I get quite a few emails and and private messages that contain the phrase above, “Network is down! Please help!”. In the vast majority of cases it usually turns out that the problem is self inflicted. Here are a few simple steps that I strongly recommend you look at if it’s your responsibility to manage the network. These features will help mitigate some serious operational issues from spoiling your day.

Spanning Tree (FastStart / PortFast)

You can’t imagine how many times I’ve seen a patch cable patched from one port on a switch in an IDF back into the same switch. It’s usually a case of sloppy cabling, although I’ve met a quite a few users who thought that both ends of the cable needed to be plugged into the wall plate. In order to prevent such scenarios from spoiling your day you should verify that you have Spanning Tree enabled on all edge ports (ports that face the users). And you should also verify that you have FastStart or PortFast enabled to help eliminate the 30 second delay during STP learning (those of us that used to run IPX/SPX networks with Novell NetWare learned this a long time ago).

BPDU Guard (called BPDU Filtering by some vendors)

In addition to Spanning Tree you should also enable BPDU guard on all your edge ports. This will immediately disable the switchport if it receives a BPDU Spanning Tree frame. This can help mitigate topology loops and it can help prevent your users from plugging in their own switches in their offices or cubicles. There are some scenarios where connecting the PC port on the back of an Avaya IP phone could potentially put a loop into the network. BPDU guard (called BPDU filtering by Avaya) will detect the BPDU frame and disable the port, thereby protecting the network.

Rate Limiting (Broadcast / Multicast)

If your in the networking field long enough you’ll eventually see some really odd behavior from one or more devices. I’ve seen my fair share of oddities and eventually you’ll run across a few too. You should prepare for that eventuality today by configuring broadcast and Multicast rate limiting on your edge ports. This helps prevent any single device from flooding too many broadcast or Multicast packets into your network which could impact either other devices connected to the network or the network itself.

DHCP Snooping

Dynamic Host Configuration Protocol (DHCP) snooping provides security to the network by preventing DHCP spoofing. DHCP spoofing refers to an attacker’s ability to respond to DHCP requests with false IP information. DHCP snooping acts like a firewall between untrusted hosts and the DHCP servers, so that DHCP spoofing cannot occur. There are 2 types of ports in DHCP snooping, trusted and untrusted. The network will only allow DHCP responses from trusted ports – usually uplinks as opposed to allowing DHCP responses from untrusted ports – usually edge switch ports.

DHCP Option 82 – With DHCP Option 82 the switch will append additional information to the DHCP request which can be stored and captured by your DHCP server (or your IP Address Management solution) to help identify the switch and port from which the request was initiated.

ARP Inspection

Without dynamic ARP inspection, a malicious user can attack hosts, switches, and routers connected to the Layer 2 network by poisoning the ARP caches of systems connected to the subnet and by intercepting traffic intended for other hosts on the subnet. Dynamic ARP inspection prevents this type of attack. It intercepts, logs, and discards ARP packets with invalid IP-to-MAC address bindings.

The address binding table is dynamically built from information gathered in the DHCP request and reply when DHCP snooping is enabled. The MAC address from the DHCP request is paired with the IP address from the DHCP reply to create an entry in the DHCP binding table.

When you enable Dynamic ARP inspection, ARP packets on untrusted ports are filtered based on the source MAC and IP addresses stored in the DHCP snooping table. The switch forwards an ARP packet when the source MAC and IP address matches an entry in the DHCP snooping table. Otherwise, the ARP packet is dropped.

IP Source Guard

IP Source Guard provides security to the network by filtering clients with invalid or spoofed IP addresses. IP Source Guard is a Layer 2 (L2), port-to-port feature that works closely with information in the Dynamic Host Control Protocol (DHCP) snooping binding table. When you enable IP Source Guard on an untrusted port with DHCP snooping enabled, an IP filter entry is created or deleted for that port automatically, based on IP information stored in the corresponding DHCP binding table entry. When a connecting client receives a valid IP address from the DHCP server, a filter is installed on the port to allow traffic only from the assigned IP address. A maximum of 10 IP addresses are allowed on each IP Source Guard-enabled port. When this number is reached, no more filters are set up and traffic is dropped.

The APs will try to associate via a Layer 2 broadcast with a controller, if they fail to adopt via Layer 2 they will issue a a DHCP request with the vendor class IDs listed above.

This is really helpful in your IPAM or DHCP server, you can define specific pools based on the vendor class to return very specific DHCP options. In this case you would probably want to return DHCP option 189 (string) with the IP address of the wireless LAN switch. You can include multiple IPs separated by commas.

Here are three features that are often not implemented in most networks but could help to really address security shortcomings and potential operational issues.

I’ve recently started implementing DHCP Snooping, Dynamic ARP Inspection and IP Source Guard in my networks so I thought I’d take the time to strike up a conversation around those features and hear what everyone else is doing.

These features are all intended to provide a more stable network environment and help protect against intentional and unintentional events that can interfere with the proper operation of the network. Before we can implement any of these features we need to understand how they work and what changes (if any) will be needed to support them.

Hopefully most people have Spanning Tree, FastStart, BPDU filtering (guard) and Broadcast/Multicast rate limiting enabled by now but if you don’t you should certainly look at enabling and configuring those features before you start to try any of the ones I’m about to discuss.

DHCP Snooping

Dynamic Host Configuration Protocol (DHCP) snooping provides security to the network by preventing DHCP spoofing. DHCP spoofing refers to an attacker’s ability to respond to DHCP requests with false IP information. DHCP snooping acts like a firewall between untrusted hosts and the DHCP servers, so that DHCP spoofing cannot occur. There are 2 types of ports in DHCP snooping, trusted and untrusted. The network will only allow DHCP responses from trusted ports – usually uplinks as opposed to allowing DHCP responses from untrusted ports – usually edge switch ports.

Example, if someone brings a Linksys router into the office and connects the LAN side to your network the Linksys router will start handing out 192.168.1.x IP DHCP addresses which will likely cause some chaos. DHCP Snooping will prevent those DHCP responses from poisoning your network. I’m happy to admit that this is happened to me on more than one occasion, usually at our remote finance office where auditors think their second job could be in Information Technology.

DHCP Option 82 – With DHCP Option 82 the switch will append additional information to the DHCP request which can be stored in your IPAM (IP Address Management) solution to help identify the switch and port from which the request was initiated. This can provide diagnostic and troubleshooting information which can all be stored directly in your IPAM if it supports DHCP Option 82. Infoblox supports DHCP Option 82.

The latest software releases support the ability to copy the DHCP binding table up to a TFTP server. This prevents the switch from throwing away all the previously learned DHCP transactions between reboots or restarts of the switch.

In this example our uplinks are 1/24 and 2/24, all interfaces are untrusted by default.

You can also edit the text string the switch will append per port with the following;

interface FastEthernet 1
ip dhcp-relay option82-subscriber-id

Issues with DHCP Snooping

While you need to properly identify trusted and untrusted ports there are very few draw backs to utilizing DHCP snooping. It only prevents DHCP replies from untrusted ports essentially preventing DHCP spoofing in the network. DHCP Snooping can be enabled in most networks with a few configuration changes and very little danger (impact).

Dynamic ARP Inspection

Without dynamic ARP inspection, a malicious user can attack hosts, switches, and routers connected to the Layer 2 network by poisoning the ARP caches of systems connected to the subnet and by intercepting traffic intended for other hosts on the subnet. Dynamic ARP inspection prevents this type of attack. It intercepts, logs, and discards ARP packets with invalid IP-to-MAC address bindings.

The address binding table is dynamically built from information gathered in the DHCP request and reply when DHCP snooping is enabled. The MAC address from the DHCP request is paired with the IP address from the DHCP reply to create an entry in the DHCP binding table.

When you enable Dynamic ARP inspection, ARP packets on untrusted ports are filtered based on the source MAC and IP addresses stored in the DHCP snooping table. The switch forwards an ARP packet when the source MAC and IP address matches an entry in the DHCP snooping table. Otherwise, the ARP packet is dropped.

For dynamic ARP inspection to function, DHCP snooping must be globally enabled. Dynamic ARP inspection is configured on a VLAN to VLAN basis.

Issues with Dynamic ARP Inspection

Dynamic ARP Inspection relies on the information stored in the DHCP binding table (from DHCP Snooping) to validate the ARP packets it receives on untrusted ports. Any device whether it be statically configured or dynamically configured would need to appear in the DHCP binding table.

If you have a statically configured device you’ll need to manually populate the DHCP snooping table. If someone changed out a statically configured device the MAC address would most likely need to be updated in the DHCP binding table. If the DHCP binding table was accidentally cleared the switch would block IP traffic until the DHCP binding table was re-built either manually or from DHCP transactions.

This is another great reason to use manual or reserved DHCP assignments where possible if the device requires a persistent IP address.

This feature will likely create some significant administrative overhead based on the number of devices configured with a static IP address over the number of DHCP configured devices.

IP Source Guard

IP Source Guard provides security to the network by filtering clients with invalid or spoofed IP addresses. IP Source Guard is a Layer 2 (L2), port-to-port feature that works closely with information in the Dynamic Host Control Protocol (DHCP) snooping binding table. When you enable IP Source Guard on an untrusted port with DHCP snooping enabled, an IP filter entry is created or deleted for that port automatically, based on IP information stored in the corresponding DHCP binding table entry. When a connecting client receives a valid IP address from the DHCP server, a filter is installed on the port to allow traffic only from the assigned IP address. A maximum of 10 IP addresses are allowed on each IP Source Guard-enabled port. When this number is reached, no more filters are set up and traffic is dropped.

Issues with IP Source Guard

IP Source Guard utilizes the information stored in the DHCP binding table (from DHCP Snooping) to validate the IP traffic. Any device whether it be statically configured or dynamically configured would need to appear in the DHCP binding table. Statically configured devices would need to be manually placed in the DHCP binding table. If someone changed out a device the MAC address would most likely need to be updated in the DHCP binding table. If the DHCP binding table was accidentally cleared the switch would block IP traffic until the DHCP binding table was re-built either manually or from DHCP transactions.

This feature will likely create some significant administrative overhead based on the number of devices configured with a static IP address over the number of DHCP configured devices.

My Thoughts

I’m taking a split approach right now, let me explain. I’ll be utilizing DHCP snooping and DHCP Option 82 throughout my network. Unfortunately due to the administrative overhead required to leverage Dynamic ARP Inspection and IP Source Guard I’m only planning on implementing those features at my smaller remote offices – the ones I usually have issues with from a security perspective. Since those remote offices are much smaller and usually don’t make too many changes it offers me the opportunity to evaluate the new features without over taxing my network administrators and engineers.

We recently migrated from Alcatel-Lucent’s VitalQIP to Infoblox for our IPAM (IP Address Management) solution. I hope to make a more detailed post reviewing Infoblox in the future, for now I’ll stick with the issue of integrating with the API interface. One of our goals for the past few years has been to enable MAC address registration essentially turning off the dynamic nature of DHCP. This would prevent someone from connecting any device to our internal network and getting a DHCP issued IP address. It certainly not a complete solution to the security dilemmas but it would be a good first step.

I do most of my work with CentOS and RedHat Linux because those are the distributions that my organization supports internally (even if I’m one of two people that support Linux across the entire organization). In this case I was working with a CentOS 5.7 server but I was having an issue compiling and installing the Infoblox Perl modules.

LWP::UserAgent version 5.813 required–this is only version 2.033

When I attempted to compile the Infoblox Perl modules I received the following errors;

LWP::UserAgent version 5.813 required--this is only version 2.033 at /usr/lib/perl5/site_perl/5.8.8/Infoblox/Agent.pm line 3.
BEGIN failed--compilation aborted at /usr/lib/perl5/site_perl/5.8.8/Infoblox/Agent.pm line 3.
Compilation failed in require at /usr/lib/perl5/site_perl/5.8.8/Infoblox/Session.pm line 19.
BEGIN failed--compilation aborted at /usr/lib/perl5/site_perl/5.8.8/Infoblox/Session.pm line 19.
Compilation failed in require at /usr/lib/perl5/site_perl/5.8.8/Infoblox.pm line 8.
BEGIN failed--compilation aborted at /usr/lib/perl5/site_perl/5.8.8/Infoblox.pm line 8.
Compilation failed in require at ./ibcli.pl line 78.
BEGIN failed--compilation aborted at ./ibcli.pl line 78.

This was with Perl 5.8.8 on CentOS 5.7 x64, unfortunately it seems this is a known issue with the version of LWP::UserAgent that is currently being distributed via the CentOS repository.

I was able to spin up a new CentOS 6.0 x86 server which was running Perl 5.10.1 and didn’t experience this problem.

The installation was pretty straight forward (except for the issue above) but the API reference manual does a very thorough job of detailing all the possible installation methods on both Windows and Unix/Linux. I just opened a browser to one of the Infoblox appliances and downloaded the Perl modules.

https://10.1.1.1/api/dist/CPAN/authors/id/INFOBLOX/

Just replace the IP address of 10.1.1. with the IP address of your Infoblox appliance. I’m not sure why Infoblox hides their manuals behind their support portal, I just don’t understand why companies do that. You can find the manual right here, Infoblox_API_Documentation_6.1.0.pdf.

This is an update to a fairly old post I made a few years back now providing an example dhcpd.conf configuration file for use in an Avaya (formerly Nortel) IP telephony environment. I was recently working on a few things and discovered that the Avaya IP phones ignore the next-server option within my dhcpd.conf file. A few tests and I quickly found that I needed to define the tftp-server-name option with the IP address of the TFTP server (see the global section of the dhcpd.conf file below).

If time allows I hope to post an update in the very near future covering the new Nortel-i2004-B option string. While working with the SIP 4.0 software release in the past few articles I did learn that the Avaya SIP IP phones can utilize a new DHCP vendor class, Nortel-SIP-Phone-A which can allow you a little flexibility when configuring them via DHCP and TFTP.

In the same file below I setup four DHCP scopes; one for 2111/2212/6020/6040 wireless handsets, one for i2002/i2004/1100/1200 series IP phones, one for 1100/1200 series IP phones running SIP and one for all other devices (laptops, desktops, etc). Just a quick note about the example below, you’ll notice that I have no pools in the 192.168.1.0/24 network. All the pools are in the 192.168.25.0/24 network.