Policy routing with IP Filter on FreeBSD

In this post I’ll write about implementation of policy routing with IP Filter on FreeBSD. Policy routing is a process of forcing packets to follow a particular route not necessary through default gateway. This is very useful in a multihomed environment when your FreeBSD server acts as a router and you want different networks to be routed differently based on a source network or interface.

Let’s take a look at the following case study put into production recently.

We own a server running FreeBSD 7.0-STABLE and acting as a router for one of our projects. There are three network interfaces in this server:

tl0 — has a public IP address assigned (XXX.YYY.ZZZ.35 / 255.255.255.240). The default gateway (XXX.YYY.ZZZ.33) is from the same subnet where tl0 belongs to.

fxp0 — has a public IP address assigned as well (XXX.YYY.ZZZ.50 / 255.255.255.240). Note that this is the same network but different subnet. The next hop for this subnet is XXX.YYY.ZZZ.49.

We want to NAT 10.0.14.0 network and route all outgoing packets through fxp0 interface. tl0 will be only used for providing http services for outside visitors, but all outgoing traffic should be routed through fxp0. The kernel was compiled with IP Filter support:

%ipf -V

ipf: IP Filter: v4.1.28(404)

Kernel: IP Filter: v4.1.28

Running: yes

Log Flags: 0 = none set

Default: block all, Logging: available

Here we go with the implementation.

%cat/etc/rc.conf

gateway_enable="YES"

defaultrouter="XXX.YYY.ZZZ.33"

ifconfig_tl0="inet XXX.YYY.ZZZ.35 netmask 255.255.255.240"

ifconfig_fxp0="inet XXX.YYY.ZZZ.50 netmask 255.255.255.240"

ifconfig_fxp1="inet 10.0.14.1 netmask 255.255.255.0"

ipfilter_enable="YES"

ipfilter_rules="/etc/ipf.rules"

ipmon_enable="YES"

ipmon_flags="-Ds"

ipnat_enable="YES"

ipnat_rules="/etc/ipnat.conf"

%cat/etc/ipnat.conf

map fxp0 10.0.14.0/24 –>0/32 portmap tcp/udp 30000:60000

map fxp0 10.0.14.0/24 –>0/32

This is how our routing table looks like:

%netstat -rn

Routing tables

Internet:

Destination Gateway Flags Refs Use Netif Expire

default XXX.YYY.ZZZ.33 UGS 040285 tl0

10.0.14.0/24link#3 UC 0 0 fxp1

10.0.14.255 ff:ff:ff:ff:ff:ff UHLWb 17 fxp1

XXX.YYY.ZZZ.32/28link#1 UC 0 0 tl0

XXX.YYY.ZZZ.3300:09:7c:61:93:30 UHLW 21196 tl0 1118

XXX.YYY.ZZZ.3500:50:8b:50:f2:0e UHLW 13271 lo0

XXX.YYY.ZZZ.48/28link#2 UC 0 0 fxp0

XXX.YYY.ZZZ.5000:02:b3:eb:cc:24 UHLW 13 lo0

127.0.0.1127.0.0.1 UH 02193 lo0

IP Filter is configured with basic ruleset to protect tl0 and fxp0 from outside intrusions and all outgoing traffic is allowed from fxp1.

So far so good, but, it didn’t work. 10.0.14/24 users are not able to access outside resources, you don’t see anything NAT-ed on fxp0 interface and traceroute from 10.0.14/24 terminates on fxp1 interface:

%ipnat -l

List of active MAP/Redirect filters:

map fxp0 10.0.14.0/24 –>0.0.0.0/32 portmap tcp/udp 30000:60000

map fxp0 10.0.14.0/24 –>0.0.0.0/32

List of active sessions:

Here comes the beauty of policy routing. We need to add the following rule to our IP Filter ruleset:

pass in quick on fxp1 from 10.0.14.0/24 to 10.0.14.1

pass in quick on fxp1 to fxp0:XXX.YYY.ZZZ.49 from 10.0.14.0/24 to any keep state

The first rule will allow to utilize whatever resources you have configured on fxp1 interface without any redirecting. We use dnscache running on fxp1 and don’t want to forward incoming DNS traffic.

The second rule literally means the following. Whatever traffic hitting fxp1 interface and coming from 10.0.14.0/24 network should be redirected or forwarded to fxp0 interface with the gateway set to XXX.YYY.ZZZ.49. And finally, we do keep state of the traffic to get it back.