Windows and Extension mechanisms for DNS (EDNS)

Introduction

RFC 2671 defines the Extension mechanisms for DNS (EDNS), or EDNS0 as it is also known. EDNS is a specification for expanding the size of several parameters of the Domain Name System (DNS) protocol. More info about EDNS here. As of Windows Server 2008 R2, the Windows DNS service will use EDNS in its queries to other DNS servers. Because of the way EDNS works, this should not present a problem. EDNS only adds data to the end of a DNS packet. Any server not supporting EDNS will just ignore those extra bytes and process the first 512 bytes as a regular DNS packet. Unfortunately this is not always so. In some cases, I cannot explain why, DNS servers reject or return an error when they receive an EDNS packet. This could possibly be because the receiving DNS server has some intrusion/hacking functionality in place which causes it to reject a packet longer than 512 bytes, or that it supports the EDNS specification partially. Whichever it is, this presents a problem. The best way to fix it is to update the offending DNS server. This should be easy to do and solve the problem for everybody. If that is not possible we have to disable the EDNS functionality on the Windows DNS servers. To do this you use dnscmd.exe.

To see if EDNS is active on your DNS server:

dnscmd.exe /info /EnableEdnsProbes

If this returns 1 EDNS is active.

To disable EDNS on your DNS server:

dnscmd.exe <servername> /Config /EnableEDNSProbes 0

More information

The answer received from a non-compliant DNS server when queried using EDNS could generate event 5504 in the DNS log of the Windows DNS server that sent the query. This is an example:
Message : The DNS server encountered an invalid domain name in a packet from <IP address of DNS server>. The packet will be rejected. The event data contains the DNS packet.

It seems that any Windows DNS server upgraded to Windows Server 2008 R2 retain its original settings and does not use EDNS unless specifically configured to do so.

A network trace can reveal if a DNS query uses EDNS. This example is from Wireshark:

Notice the Type: OPT (EDNS0 option). This means the DNS packet uses EDNS. A packet that does not use EDNS will not have any Addition records.

Firewall may sometimes interfere with EDNS packets. The original DNS specification sets a limit of 512 bytes on DNS packet. If a firewall encounters a packet that is longer than 512 bytes (this would be an EDNS packet) it may block it, causing DNS resolution to fail.

In conclusion

The EDNS draft was first published in 1999, and for good reason. DNS is one of the corner stones of the Internet and we really need the functionality provided by EDNS (DNSSEC, for instance, requires EDNS to work). In my mind there is no reason why anyone should not support it, be it DNS servers or firewalls. Microsoft should not be blamed for turning on functionality that we have had almost 10 years to implement.