Archive for the ‘VLAN’ tag

A friend recently presented me with the following challenge: Configure a system through which several appliances, all of them having an identical, non-routable, default IP configuration, can be reached simultaneously. All appliances are preconfigured with an IP address of 192.168.1.1 and they had no routing configuration enabled. Yet they should all be reachable over the Internet. The diagram below shows the basic infrastructure.

For the server (the nice grey cabinet to the left) to reach all those systems at the same time, a VLAN setup was required, so we set up a VLAN trunk to a nearby switch. Then the different appliances were connected to different un-tagged switch ports in dedicated VLANs – the first appliance was connected to a switch port in VLAN 10, the next to a switch port in VLAN 11, etc. Corresponding VLAN interfaces were configured on the server, and a local IP address was configured on each. To keep a minimum level of sanity, I configured the VLAN interface on vlan10 as 192.168.1.10, while vlan11 got 192.168.1.11, etc. To avoid too much confusion on the switch, I also set the MAC address on each VLAN interface to distinct values.

At this point I could reach the different appliances from the different source IPs:# ping -I 192.168.1.10 192.168.1.1 -c1
PING 192.168.1.1 (192.168.1.1) from 192.168.1.10 : 56(84) bytes of data.
64 bytes from 192.168.1.1: icmp_seq=1 ttl=64 time=0.119 ms

Snooping with tcpdump on the different VLAN interfaces, also showing the MAC addresses, confirmed that the pings were reaching the correct appliance.

Then I set up corresponding IP addresses on the outside interface (eth0). The 10.0.0.195 address matches the device in VLAN 10, 10.0.0.196 matches VLAN 11 and so on. The outside IP addresses are official IP addresses, but I’ve changed them to protect the innocent.

The above commands maps the outside addresses on eth0 to the corresponding VLANs on the inside. Note that every incoming packet is destination NATed (DNAT) to 192.168.1.1 while they are being source NATed (SNAT) based on the VLAN to which they are mapped. As the appliances were not able to route traffic, all communication had to look like it happened on the local network only.

Now I expected to be able to access each appliance from the outside, but it turned out that only one appliance could be reached. tcpdumping on each VLAN interface confirmed that traffic from the outside reached each appliance, and the appliance responded, but the response never came all the way back. This was rather puzzling as nothing appeared in any log. However, my colleagues reminded me about rp_filter, which dropped weird traffic without logging anything. Thus, the key that unlocked everything was this:echo 0 > /proc/sys/net/ipv4/conf/all/rp_filter
echo 0 > /proc/sys/net/ipv4/conf/eth1.10/rp_filter
echo 0 > /proc/sys/net/ipv4/conf/eth1.11/rp_filter

And voila – from the outside world, we were able to reach every single appliance on the inside, all with the same unroutable RFC1918 address.

I’ve been planning to do this for a long time and now I’m finally there. My home network now consists of two virtual host servers (one Xen and one KVM) and a firewall between them, all nodes understanding VLANs. On top of this, add a small but powerful Linksys switch and a Linksys wireless access point running OpenWRT Kamikaze 8.09 (RC1). Among other things, this setup should facilitate testing Munin and other stuff without breaking the current functionality too much.