divert: Define semantics for SO_REUSEPORT_LB on divert sockets

Allow SO_REUSEPORT_LB to be set on divert sockets.  If set, then bind()
will add the socket to a "load-balancing group".  When a divert-to rule
matches a port with an associated group, the corresponding state ID is
used to select a specific socket from the group.  Packets without an
associated state are simply forwarded to the first socket in the group.
For now I only pass a state ID from pf, as I couldn't see a useful
identifier on the ipfw side.

This implementation is simple but has a caveat, that being that if
sockets are added to the group while flows are being processed, the size
of the group will change and this changes the mapping of state IDs to
sockets.  So, to get a consistent mapping, the divert socket application
must bind all of its sockets before any traffic is diverted by the
firewall.

Reviewed by:	glebius
MFC after:	1 month
Sponsored by:	OPNsense
Sponsored by:	Klara, Inc.
Differential Revision:	https://reviews.freebsd.org/D56563

This commit is contained in:

Mark Johnston

2026-05-26 12:43:51 +00:00

parent ccb14be785

commit 895a0ae67f

7 changed files with 173 additions and 25 deletions

share/man/man4/divert.4

+21 -3

View File

@@ -1,5 +1,4 @@
 .\"
 .Dd January 23, 2026
 .Dd April 21, 2026
 .Dt DIVERT 4
 .Os
 .Sh NAME
@@ -57,7 +56,26 @@ firewall processing at the next rule.
 .Pp
 By reading from and writing to a divert socket, matching packets
 can be passed through an arbitrary ``filter'' as they travel through
 the host machine, special routing tricks can be done, etc.
 the host machine, special routing tricks can be done, etc..
 .Pp
 Multiple divert sockets may be bound to the same port if the
 .Dv SO_REUSEPORT_LB
 socket option is set on all of them.
 In this case, the kernel will attempt to load-balance packets among
 the sockets.
 The implementation ensures that packets from the same flow are delivered
 to the same socket.
 To this end it relies on the firewall to provide a flow identifier with
 each diverted packet.
 When using the
 .Xr pf 4
 firewall, this is the associated state ID, if one exists, otherwise all
 packets are diverted to the first socket in the group.
 Currently the
 .Xr ipfw 4
 firewall does not provide a flow identifier, so all packets are diverted
 to the first socket in the group.
 At most 32 sockets can be bound to the same port.
 .Sh READING PACKETS
 Packets are diverted either as they are ``incoming'' or ``outgoing.''
 Incoming packets are diverted after reception on an IP interface,