Question about pf/routing - BSD

This is a discussion on Question about pf/routing - BSD ; I have a situation in which I am trying to deploy OpenBSD 3.8 as a
router/firewall between two subnets. I have the NICs configured, I
have enabled forwarding, and I have configured pf for the traffic that
I need passed. ...

Question about pf/routing

I have a situation in which I am trying to deploy OpenBSD 3.8 as a
router/firewall between two subnets. I have the NICs configured, I
have enabled forwarding, and I have configured pf for the traffic that
I need passed. However, the system seems to be dropping traffic in
certain instances.

A bit more background: This router/firewall needs to act as the
default gateway to clients on one of the subnets. However, it should
then redirect all traffic destined for the Internet to a separate
hardware firewall, and route traffic destined for internal traffic
across to the second NIC. The behavior I'm seeing is that when clients
send traffic destined for the Internet to this server, the connections
are dropped. I can ping through the OpenBSD router/firewall to the
Internet just fine, but TCP and UDP connections just don't work.

I don't think it's a routing issue, since the ping works; instead, I
think I've mucked up the pf configuration. Here's a sanitized pf.conf:

# pf.conf for lab-to-lab firewall

# Definitions
int_if = "le2"
ext_if = "le1"

tcp_services = "22"
priv_net = "172.16.22.0/24"

# Options
set block-policy return
set loginterface $ext_if
set skip on lo0

# Scrub
scrub in all
scrub out all

# NAT/Rdr
nat on $ext_if from $priv_net to any -> ($ext_if)

# Filter rules
block all

pass in on $ext_if inet proto tcp from any to ($ext_if) port
$tcp_services flags S/SA keep state

pass in on $int_if from $priv_net to any keep state
pass out on $int_if from any to $priv_net keep state

pass out on $ext_if proto tcp all modulate state flags S/SA
pass out on $ext_if proto { udp, icmp } all keep state

Sorry, I'm a bit new to pf (trying to learn my way around this very
powerful firewall). So what have I missed?

TIA.

--
Regards,
Scott Lowe
ePlus Technology, Inc.

Re: Question about pf/routing

Scott Lowe wrote:
> I have a situation in which I am trying to deploy OpenBSD 3.8 as a
> router/firewall between two subnets. I have the NICs configured, I
> have enabled forwarding, and I have configured pf for the traffic that
> I need passed. However, the system seems to be dropping traffic in
> certain instances.
>
> A bit more background: This router/firewall needs to act as the
> default gateway to clients on one of the subnets. However, it should
> then redirect all traffic destined for the Internet to a separate
> hardware firewall, and route traffic destined for internal traffic
> across to the second NIC. The behavior I'm seeing is that when clients
> send traffic destined for the Internet to this server, the connections
> are dropped. I can ping through the OpenBSD router/firewall to the
> Internet just fine, but TCP and UDP connections just don't work.
>
> I don't think it's a routing issue, since the ping works; instead, I
> think I've mucked up the pf configuration. Here's a sanitized pf.conf:
>
> # pf.conf for lab-to-lab firewall
>
> # Definitions
> int_if = "le2"
> ext_if = "le1"
>
> tcp_services = "22"
> priv_net = "172.16.22.0/24"
>
> # Options
> set block-policy return
> set loginterface $ext_if
> set skip on lo0
>
> # Scrub
> scrub in all
> scrub out all
>
> # NAT/Rdr
> nat on $ext_if from $priv_net to any -> ($ext_if)
>
> # Filter rules
> block all
>
> pass in on $ext_if inet proto tcp from any to ($ext_if) port
> $tcp_services flags S/SA keep state
>
> pass in on $ext_if inet proto icmp all icmp-type 8 code 0 keep state
> pass out on $ext_if inet proto icmp all icmp-type 8 code 0 keep state
> pass in on $ext_if inet proto icmp all icmp-type 0 code 0 keep state
> pass out on $ext_if inet proto icmp all icmp-type 0 code 0 keep state
> pass out on $int_if inet proto icmp all icmp-type 8 code 0 keep state
> pass out on $int_if inet proto icmp all icmp-type 0 code 0 keep state
>
> pass in on $int_if from $priv_net to any keep state
> pass out on $int_if from any to $priv_net keep state
>
> pass out on $ext_if proto tcp all modulate state flags S/SA
> pass out on $ext_if proto { udp, icmp } all keep state
>
> Sorry, I'm a bit new to pf (trying to learn my way around this very
> powerful firewall). So what have I missed?

Looks okay. Have you configured forwarding (net.inet.ip{,6}.forward)?

Tightening the configuration a little might be warranted, though - the
above configuration is more permissive than necessary.

And flags S/SA is redundant with scrub (which will normalize traffic,
including dropping TCP packets of a type that should not be sent
considering the connection state). In fact, 'pass out on $ext_if
proto { tcp, udp, icmp } modulate state' will do pretty much what the
second 'paragraph' from below does.

(Note that 'pass out on $ext_if modulate state' would pass, for
instance, IPsec traffic too.)

Joachim

Re: Question about pf/routing

On 2006-04-05 04:29:13 -0400, jKILLSPAM.schipper@math.uu.nl said:
> Scott Lowe wrote:
>> I have a situation in which I am trying to deploy OpenBSD 3.8 as a
>> router/firewall between two subnets. I have the NICs configured, I
>> have enabled forwarding, and I have configured pf for the traffic that
>> I need passed. However, the system seems to be dropping traffic in
>> certain instances.
>>
>> A bit more background: This router/firewall needs to act as the
>> default gateway to clients on one of the subnets. However, it should
>> then redirect all traffic destined for the Internet to a separate
>> hardware firewall, and route traffic destined for internal traffic
>> across to the second NIC. The behavior I'm seeing is that when clients
>> send traffic destined for the Internet to this server, the connections
>> are dropped. I can ping through the OpenBSD router/firewall to the
>> Internet just fine, but TCP and UDP connections just don't work.
>>
>> I don't think it's a routing issue, since the ping works; instead, I
>> think I've mucked up the pf configuration. Here's a sanitized pf.conf:
>>
>> # pf.conf for lab-to-lab firewall
>>
>> # Definitions
>> int_if = "le2"
>> ext_if = "le1"
>>
>> tcp_services = "22"
>> priv_net = "172.16.22.0/24"
>>
>> # Options
>> set block-policy return
>> set loginterface $ext_if
>> set skip on lo0
>>
>> # Scrub
>> scrub in all
>> scrub out all
>>
>> # NAT/Rdr
>> nat on $ext_if from $priv_net to any -> ($ext_if)
>>
>> # Filter rules
>> block all
>>
>> pass in on $ext_if inet proto tcp from any to ($ext_if) port
>> $tcp_services flags S/SA keep state
>>
>> pass in on $ext_if inet proto icmp all icmp-type 8 code 0 keep state
>> pass out on $ext_if inet proto icmp all icmp-type 8 code 0 keep state
>> pass in on $ext_if inet proto icmp all icmp-type 0 code 0 keep state
>> pass out on $ext_if inet proto icmp all icmp-type 0 code 0 keep state
>> pass out on $int_if inet proto icmp all icmp-type 8 code 0 keep state
>> pass out on $int_if inet proto icmp all icmp-type 0 code 0 keep state
>>
>> pass in on $int_if from $priv_net to any keep state
>> pass out on $int_if from any to $priv_net keep state
>>
>> pass out on $ext_if proto tcp all modulate state flags S/SA
>> pass out on $ext_if proto { udp, icmp } all keep state
>>
>> Sorry, I'm a bit new to pf (trying to learn my way around this very
>> powerful firewall). So what have I missed?
>
> Looks okay. Have you configured forwarding (net.inet.ip{,6}.forward)?
>
> Tightening the configuration a little might be warranted, though - the
> above configuration is more permissive than necessary.
>
> And flags S/SA is redundant with scrub (which will normalize traffic,
> including dropping TCP packets of a type that should not be sent
> considering the connection state). In fact, 'pass out on $ext_if
> proto { tcp, udp, icmp } modulate state' will do pretty much what the
> second 'paragraph' from below does.
>
> (Note that 'pass out on $ext_if modulate state' would pass, for
> instance, IPsec traffic too.)
>
> Joachim

I have not configure IPv6 forwarding, but I have configured IPv4 forwarding.

I'm not terribly concerned about the overall security of this
configuration, since this is an internal firewall/router intended to
segregate a test subnet from a corporate subnet. What I am concerned
about, however, is the fact that there is currently some sort of
routing/configuration error that prevents PCs on the test subnet from
routing through this firewall/router to a different firewall onto the
Internet.

For example, let's say this router/firewall is using 172.16.1.1 as the
IP address for its internal interface, and there is another firewall
(connected to the Internet) using 172.16.1.254 as the IP address for
its internal interface. If a PC uses 172.16.1.1 (the OpenBSD machine)
as its default gateway, it is able to ping onto the Internet but
TCP/UDP connections fail. So there's something about this
configuration that is interfering with this machine's ability to
receive traffic on the internal interface and then redirect that
traffic to the firewall that connects to the Internet. Does that make
sense?

--
Regards,
Scott Lowe
ePlus Technology, Inc.

Re: Question about pf/routing

Scott Lowe wrote:
> On 2006-04-05 04:29:13 -0400, jKILLSPAM.schipper@math.uu.nl said:
>
>> Scott Lowe wrote:
>>> I have a situation in which I am trying to deploy OpenBSD 3.8 as a
>>> router/firewall between two subnets. I have the NICs configured, I
>>> have enabled forwarding, and I have configured pf for the traffic that
>>> I need passed. However, the system seems to be dropping traffic in
>>> certain instances.
>>>
>>> A bit more background: This router/firewall needs to act as the
>>> default gateway to clients on one of the subnets. However, it should
>>> then redirect all traffic destined for the Internet to a separate
>>> hardware firewall, and route traffic destined for internal traffic
>>> across to the second NIC. The behavior I'm seeing is that when clients
>>> send traffic destined for the Internet to this server, the connections
>>> are dropped. I can ping through the OpenBSD router/firewall to the
>>> Internet just fine, but TCP and UDP connections just don't work.
>>>
>>> I don't think it's a routing issue, since the ping works; instead, I
>>> think I've mucked up the pf configuration. Here's a sanitized pf.conf:
>>>
>>> # pf.conf for lab-to-lab firewall
>>>
>>> # Definitions
>>> int_if = "le2"
>>> ext_if = "le1"
>>>
>>> tcp_services = "22"
>>> priv_net = "172.16.22.0/24"
>>>
>>> # Options
>>> set block-policy return
>>> set loginterface $ext_if
>>> set skip on lo0
>>>
>>> # Scrub
>>> scrub in all
>>> scrub out all
>>>
>>> # NAT/Rdr
>>> nat on $ext_if from $priv_net to any -> ($ext_if)
>>>
>>> # Filter rules
>>> block all
>>>
>>> pass in on $ext_if inet proto tcp from any to ($ext_if) port
>>> $tcp_services flags S/SA keep state
>>>
>>> pass in on $ext_if inet proto icmp all icmp-type 8 code 0 keep state
>>> pass out on $ext_if inet proto icmp all icmp-type 8 code 0 keep state
>>> pass in on $ext_if inet proto icmp all icmp-type 0 code 0 keep state
>>> pass out on $ext_if inet proto icmp all icmp-type 0 code 0 keep state
>>> pass out on $int_if inet proto icmp all icmp-type 8 code 0 keep state
>>> pass out on $int_if inet proto icmp all icmp-type 0 code 0 keep state
>>>
>>> pass in on $int_if from $priv_net to any keep state
>>> pass out on $int_if from any to $priv_net keep state
>>>
>>> pass out on $ext_if proto tcp all modulate state flags S/SA
>>> pass out on $ext_if proto { udp, icmp } all keep state
>>>
>>> Sorry, I'm a bit new to pf (trying to learn my way around this very
>>> powerful firewall). So what have I missed?
>>
>> Looks okay. Have you configured forwarding (net.inet.ip{,6}.forward)?
>>
>> Tightening the configuration a little might be warranted, though - the
>> above configuration is more permissive than necessary.
>>
>> And flags S/SA is redundant with scrub (which will normalize traffic,
>> including dropping TCP packets of a type that should not be sent
>> considering the connection state). In fact, 'pass out on $ext_if
>> proto { tcp, udp, icmp } modulate state' will do pretty much what the
>> second 'paragraph' from below does.
>>
>> (Note that 'pass out on $ext_if modulate state' would pass, for
>> instance, IPsec traffic too.)
>
> I have not configure IPv6 forwarding, but I have configured IPv4 forwarding.

Okay, should work.
> I'm not terribly concerned about the overall security of this
> configuration, since this is an internal firewall/router intended to
> segregate a test subnet from a corporate subnet.

Well, it's your network. Be sure to properly separate test/production,
though.
> What I am concerned
> about, however, is the fact that there is currently some sort of
> routing/configuration error that prevents PCs on the test subnet from
> routing through this firewall/router to a different firewall onto the
> Internet.
>
> For example, let's say this router/firewall is using 172.16.1.1 as the
> IP address for its internal interface, and there is another firewall
> (connected to the Internet) using 172.16.1.254 as the IP address for
> its internal interface. If a PC uses 172.16.1.1 (the OpenBSD machine)
> as its default gateway, it is able to ping onto the Internet but
> TCP/UDP connections fail. So there's something about this
> configuration that is interfering with this machine's ability to
> receive traffic on the internal interface and then redirect that
> traffic to the firewall that connects to the Internet. Does that make
> sense?

This might have something to do with the fact that both firewalls are on
the same subnet, address-wise, but not physically. OTOH, I do not
immediately see how that could lead to trouble, nor do I see how ICMP
could work in such a situation - as you said, it doesn't *look* like
routing problems.

Truly strange. Presuming you have enabled pf(4), and have loaded the
above ruleset minus errors introduced by line wrapping, I do not see how
this would not work.

Could you post the output of tcpdump -nvvvi $int_if and tcpdump -nvvvi
$ext_if, when you do each of pinging another IP, connecting via UDP, and
connecting via TCP?

Also, a general good rule of debugging pf - set 'block log all' (instead
of just 'block log') and have tcpdump listen to pflog0. Anything that
appears there is blocked by pf(4) - this is good for ruling out pf as
the culprit, or debugging a ruleset.

Finally, what are you running beyond your firewall? ISTR some problems
with Windows IP stacks and 'scrub' without 'no-df' (the reason being
MSisms, of course).

Joachim

Re: Question about pf/routing

On 2006-04-05 11:31:07 -0400, jKILLSPAM.schipper@math.uu.nl said:
> Scott Lowe wrote:
>>>> # pf.conf for lab-to-lab firewall
>>>>
>>>> # Definitions
>>>> int_if = "le2"
>>>> ext_if = "le1"
>>>>
>>>> tcp_services = "22"
>>>> priv_net = "172.16.22.0/24"
>>>>
>>>> # Options
>>>> set block-policy return
>>>> set loginterface $ext_if
>>>> set skip on lo0
>>>>
>>>> # Scrub
>>>> scrub in all
>>>> scrub out all
>>>>
>>>> # NAT/Rdr
>>>> nat on $ext_if from $priv_net to any -> ($ext_if)
>>>>
>>>> # Filter rules
>>>> block all
>>>>
>>>> pass in on $ext_if inet proto tcp from any to ($ext_if) port
>>>> $tcp_services flags S/SA keep state
>>>>
>>>> pass in on $ext_if inet proto icmp all icmp-type 8 code 0 keep state
>>>> pass out on $ext_if inet proto icmp all icmp-type 8 code 0 keep state
>>>> pass in on $ext_if inet proto icmp all icmp-type 0 code 0 keep state
>>>> pass out on $ext_if inet proto icmp all icmp-type 0 code 0 keep state
>>>> pass out on $int_if inet proto icmp all icmp-type 8 code 0 keep state
>>>> pass out on $int_if inet proto icmp all icmp-type 0 code 0 keep state
>>>>
>>>> pass in on $int_if from $priv_net to any keep state
>>>> pass out on $int_if from any to $priv_net keep state
>>>>
>>>> pass out on $ext_if proto tcp all modulate state flags S/SA
>>>> pass out on $ext_if proto { udp, icmp } all keep state
>>>>
>>
>> I have not configure IPv6 forwarding, but I have configured IPv4 forwarding.
>
> Okay, should work.
>
>> I'm not terribly concerned about the overall security of this
>> configuration, since this is an internal firewall/router intended to
>> segregate a test subnet from a corporate subnet.
>
> Well, it's your network. Be sure to properly separate test/production,
> though.
>
>> What I am concerned about, however, is the fact that there is currently
>> some sort of routing/configuration error that prevents PCs on the test
>> subnet from routing through this firewall/router to a different
>> firewall onto the Internet.
>>
>> For example, let's say this router/firewall is using 172.16.1.1 as the
>> IP address for its internal interface, and there is another firewall
>> (connected to the Internet) using 172.16.1.254 as the IP address for
>> its internal interface. If a PC uses 172.16.1.1 (the OpenBSD machine)
>> as its default gateway, it is able to ping onto the Internet but
>> TCP/UDP connections fail. So there's something about this
>> configuration that is interfering with this machine's ability to
>> receive traffic on the internal interface and then redirect that
>> traffic to the firewall that connects to the Internet. Does that make
>> sense?
>
> This might have something to do with the fact that both firewalls are on
> the same subnet, address-wise, but not physically. OTOH, I do not
> immediately see how that could lead to trouble, nor do I see how ICMP
> could work in such a situation - as you said, it doesn't *look* like
> routing problems.
>
> Truly strange. Presuming you have enabled pf(4), and have loaded the
> above ruleset minus errors introduced by line wrapping, I do not see how
> this would not work.
>
> Could you post the output of tcpdump -nvvvi $int_if and tcpdump -nvvvi
> $ext_if, when you do each of pinging another IP, connecting via UDP, and
> connecting via TCP?
>
> Also, a general good rule of debugging pf - set 'block log all' (instead
> of just 'block log') and have tcpdump listen to pflog0. Anything that
> appears there is blocked by pf(4) - this is good for ruling out pf as
> the culprit, or debugging a ruleset.
>
> Finally, what are you running beyond your firewall? ISTR some problems
> with Windows IP stacks and 'scrub' without 'no-df' (the reason being
> MSisms, of course).
>
> Joachim

I have isolated the issue. I don't understand *why* it's an issue,
yet, but I have isolated the issue.

In the original pf.conf, I had these rules for the internal interface:

pass in on $int_if from $priv_net to any keep state
pass out on $int_if from any to $priv_net keep state

(Along with some other ICMP specific rules earlier on as well.) This
would cause all TCP and UDP connections to the Internet to fail. After
removing some of the ICMP-specific rules as well, ICMP to the Internet
also failed.

However, with a simple change of one rule:

pass in on $int_if from $priv_net to any keep state
pass out on $int_if from any to any keep state

Now the firewall/router works as expected, and I am able to both route
traffic to the corporate network (which is the network attached to the
external interface) as well as to the Internet (which is a separate
firewall on the same subnet as the internal interface).

Any idea why that change makes it work?

(BTW, now that I at least have it working, I can tighten down the rules
a bit to ensure, as you suggested, proper protection between
test/development and production.)