Black Hole Routers

Got your attention? I like this topic because it sounds like something from outer space =) MS KB 314825 explains a curious issue that you may encounter on your networks and how to identify a black hole router.

When a network router receives a packet that is larger than the size of the Maximum Transmission Unit (MTU) of the next segment of a communications network, and that packet’s IP layer "don’t fragment" bit is flagged, the router is expected to send an ICMP "destination unreachable" message back to the sending host. If the router does not send a message, the packet might be dropped, causing a variety of errors that vary with the program that is communicating over the unsuccessful link. (These errors do not occur if a program connects to a computer on a local subnet.) The behavior may seem intermittent, but closer examination shows that the behavior can be reproduced, for example, by having a client read a large file that is sent from a remote host.

The largest buffer that can be sent unfragmented is equal to the smallest MTU that exists along a route, minus the IP and ICMP headers (in other words, the smallest MTU minus 28). For example, Ethernet has an MTU of 1,500 bytes, so under the best circumstances, the Ping utility can echo an unfragmented packet, plus an ICMP buffer, of 1,472 bytes (1,500 minus 28). The syntax for the ping command in this case is:

ping computer_name or IP_address -f -l 1472

For all local IP addresses, the expected results are as follows:

If the MTU of every segment of a routed connection is at least 1,500, the packet is successfully returned.

If there are intermediate segments that have smaller MTUs, and the routers return the appropriate ICMP "destination unreachable" packet, the Ping utility displays the message, "Packet needs to be fragmented but DF set."

If there are intermediate segments that have smaller MTUs, and the routers do not return the appropriate ICMP "destination unreachable" packet, the Ping utility displays the message, "Request timed out."

By increasing the -l parameter on successive pings, you can identify how large an unfragmented packet can travel a specific route. The smallest MTU that is in general use is 576 bytes, so you can safely start with an ICMP buffer of 548 and then work up from there. For example, if the command Ping computer_name or IP_address -f -l 972 returns packets but Ping computer_name or IP_address -f -l 973 does not return packets, the largest MTU on that route is 1,000 (972 plus 28).

KB 314825 describes a few methods for fixing this issue. I just like it because it is a cool way of using the ping utility =)