In the process we’ve learned perhaps more than we had wanted to about the behaviour of Flash engines, Apache web servers and FreeBSD system tuning, and also learned much more than we had anticipated about the finer details of Google’s online ad presentation behaviour. But one thing we did not see in all of this was any large scale jumps in the level of client use of DNSSEC validation over this period at the start of the year.

This apparent slowness in the adoption of DNSSEC a source of some frustration. We have heard many times that some of the more insidious threats to the security and integrity of the Internet’s service environment start with attacks on the DNS. Such attacks exploit a real weakness in the behaviour of many users: You type in a URL, and you see a familiar screen in response and you think you are now connected to the service you specified. But there is the risk that you are not, and this risk is there irrespective of whether the service is “secure” or not. There is a risk from various forms of so-called “man-in-the-middle” attacks that you have been mislead. Most such attacks pass largely unremarked, if not unnoticed. But from time to time the issue generates a high level of public prominence, such as the attacks that resulted from Diginotar attack on a Domain Name Certification Authority back in 2011 and the subsequent exploitation of that attack in a consequent structured attack on Gmail users located in Iran.

The best defence against these forms of attack that we’ve been able to devise is to secure the DNS, so that a system making a query of the DNS can be assured that the answer that they are given in response to their query is precisely the same information that was entered into the DNS by the authorized zone administrator. This, in turn, allows the embedding of information about domain name certification into the DNS in a secure manner (as described in RFC6394), which is a relatively effective response to the form of man-in-the-middle attack we saw as a consequence of the compromise of Diginotar’s CA services. The combination of these two measures, signing the domain name using DNSSEC, and placing certificate credentials into the DNS as signed data, would allow a user’s browser to reliably avoid using a compromised CA to validate a fake Domain Name Certificate.

For some years attention has been focussed to the effort to deploy DNSSEC in the domain name system. We have seen an extensive effort to get to a DNSSEC-signed root, and now that this has been achieved, we are now seeing this effort focus on the signing of all top level domain names (TLDs).

But signing domain names is only one half of the story. Clients’ DNS resolvers also need to retrieve this information and validate the responses they receive from the DNS. So, at the other end of the spectrum, in the realm of DNS resolvers, we have seen increasing use of DNSSEC validation, but little in the way of structured measurement of progress in this area. At APNIC Labs have been looking at ways to measure this, and are trying to answer the basic question: How many of the Internet’s user population exclusively use DNS resolvers that perform DNSSEC validation?

In late 2012 we saw some 1.6% of clients exclusively use DNSSEC-validating resolvers, using a relatively imprecise measurement methodology.

At the start of 2013 we revised the experimental technique, and saw some 3% of users appear to exclusively use DNSSEC validating resolvers. Appropriately, these users were unable to resolve a DNS name when its DNSSEC signature was invalid. In the same experiment, we also measured a further 2% of clients who use a mix of DNSSEC-validating and non-validating resolvers, so that when a DNSSEC validating resolver correctly responds with SERVFAIL the client then queries another resolver who does not perform DNSSEC validation, and therefore dutifully returns the original address record.

A couple of weeks after we conducted this experiment Google announced the inclusion of DNSSEC validation to its public DNS service. Earlier configurations of Google’s public DNS service required the client to set the DNSSEC OK (DO) flags on its queries in order to trigger a DNSSEC validation operation, but in late March Google switched this behaviour to perform a DNSSEC validation for all queries, except for those that explicitly requested no validation via the setting of the Checking Disabled (CD) flag in the DNS query.

What did we see in May when we again measured DNSSEC use by the Internet’s end user population?

Measuring DNSSEC Use

We ran an experiment across the period of the 9th May through to the 26th May, and ran a DNSSEC capability test across 2,746,777 clients, selected using an online advertisement placement method. Of these clients we saw 2,595,672 complete the experiment’s tests and submit results to our server.

As with previous DNSSEC experiments, we presented the client with three URLs, all using IPv4. URL A used a domain name that was validly DNSSEC-signed domain name. URL B had no DNSSEC signature. URL C had a DNSSEC signature that was corrupted so that DNSSEC validation would fail. All three URLs used a label part in their DNS name that was relatively. Through this measure we were trying the minimize any measurement “noise” as a result of DNS caching, ensuring that all queries were passed to the authoritative name server for both queries for the original resource records and queries for the DNSSEC DS and DNSKEY resource.

To explain “relative uniqueness” a little more, we wanted to present each client with a unique DNSSEC signed domain name. The most obvious way to achieve this would be to use a large signed zone, but there are a number of major operational considerations in maintaining a signed DNS zone file with upward of 10 million entries that would be needed to support an experiment of this scale. Instead, we opted to use a smaller pool of some 500,000 unique names, and set a one hour TTL on the DNS data. We were cycling through this name space in around 20 hours, which was considered to be comfortably in excess of the one hour TTL.

The various combination of URLs that were logged as being fetched from the associated web server produced the outcomes as shown in Table 1. This table also includes results from the earlier run of this experiment in February of this year.

URL A

URL B

URL C

Count

%

%(Feb’13)

1

no

no

no

3,040

0.1%

0.6%

2

yes

no

no

1,348

0.1%

0.1%

3

no

yes

no

4,445

0.2%

0.1%

4

yes

yes

no

216,700

8.3%

3.5%

5

no

no

yes

1,275

0.1%

0.1%

6

yes

no

yes

4,388

0.2%

0.1%

7

no

yes

yes

5,044

0.2%

0.2%

8

yes

yes

yes

2,359,462

90.9%

95.3%

Table 1 – DNSSEC Fetch Results (Web)

If a client supports DNSSEC validation outcomes then it would fetch URLs A and B, but not URL C (row 4 in Table 1). If it does not support DNSSEC validation then it should fetch all three URLs, as in all other respects the three URLs are functionally identical (row 8 in Table 1). However, some 3% of clients fetch various combinations of URLs other than these two anticipated combinations, showing that there is some variability in the precise mode of execution of the experiment in certain client scenarios.

The web server logs of this experiment show that, within an error bound of ±1%, some 8% of clients appear to be performing some kind of DNSSEC validation, based on the combinations of URLs that they are able to fetch. This represents an increase of some 5% since February, which we could surmise as being attributable to the change in behavior of the Google Public DNS resolvers.

A detailed examination of the logs of the DNS authoritative server, when coupled with the Web log data can provide a better insight into these numbers. We are looking for clients whose DNS resolvers query for both the A resource record and the DNSKEY and DS resource records, and where the client fetches the A and B URLs, but not the C URL. These clients, we assume, are using DNS resolvers that perform DNSSEC validation, and pass the client a SERVFAIL response to the DNS query associated with the C URL. Furthermore, we can assume that where a client’s DNS resolvers make no query for URL A or C’s DS or DNSKEY resource records, then we can assume that the client’s DNS resolvers perform no DNSSEC. The other case is where we see the retrieval of DNSSEC resource records, but see fetching of the C URL. We assume that in this case the client’s DNS resolvers are a mix of validating and non-validating resolvers, and in the case of the C URL the response of SERVFAIL from a DNSSEC-validating resolver causes the client to ask the same query from the next resolver in its local resolver set, or causes its DNS resolver to ask the next resolver in its DNS forwarding set without the client’s explicit knowledge.

We can combine both the DNS query profile and the web server logs to produce the results shown in Table 2.

Client Behaviour

Count

%

%
(Feb’13)

DNSSEC Validating

209,505

8.1%

3.3%

Mix of Resolvers

110,701

4.4%

2.6%

NO DNSSEC

2,206,592

87.3%

94.1%

Table 2 – DNSSEC Fetch Results (DNS)

This DNS data shows that the proportion of clients who exclusively use DNS resolvers that perform DNSSEC validation has risen by a little under 5% of the total client population, from 3.1% in February 2013 to 8.3% in May 2013.

We have also seen a significant rise in the number of clients who use a mix of DNS resolvers, some of which were seen to perform DNSSEC validation. This has risen by some 1.8%, from 2.6% in February 2013 to 4.4% in May 2013.

To what extent is this increase in the population attributable to Google’s Public DNS servers?

Of the 2.34M unique IP addresses of clients who ran this experiment, we saw 174,082 clients use Google Public DNS servers, or 7.4% of all tested clients. Of these, we saw some 128,359, or 5.5% of all clients exclusively use Google’s Public DNS servers, with the remaining 45,723 (or 2%) clients use a mix of Google DNS servers and other servers.

More specifically, where are these DNSSEC-validating clients, and what resolvers do they use?

Of those countries with more than 200 samples, those countries with the highest proportion of DNSSEC-validating clients are as follows:

Rank

CC

Count

%DNSSEC

%Mixed

%No DNSSEC

Country

1

SE

7,082

79.64

4.90

15.46

Sweden

2

SI

5,357

60.35

6.03

33.62

Slovenia

3

AG

313

57.51

8.63

33.87

Antigua and Barbuda

4

LU

734

46.59

7.08

46.32

Luxembourg

5

AO

247

42.91

21.46

35.63

Angola

6

FI

2,748

39.30

16.78

43.92

Finland

7

VN

30,273

38.41

4.06

57.53

Vietnam

8

CZ

33,005

34.97

8.27

56.76

Czech Republic

9

CL

52,924

31.22

8.45

60.33

Chile

10

JM

1,752

30.02

3.20

66.78

Jamaica

11

IE

8,985

28.79

5.49

65.72

Ireland

12

NC

290

27.93

5.52

66.55

New Caledonia

13

BB

1,476

25.81

1.69

72.49

Barbados

14

FO

242

24.79

0.83

74.38

Faroe Islands

15

UA

27,934

23.17

12.70

64.13

Ukraine

16

ID

63,288

22.84

8.06

69.10

Indonesia

17

ZA

3,333

22.47

9.72

67.81

South Africa

18

US

165,630

19.91

3.59

76.50

United States of America

19

TR

52,909

19.39

2.22

78.39

Turkey

20

AF

232

18.53

34.05

47.41

Afghanistan

2,871,753

8.73

6.93

84.34

World

Table 3 – DNSSEC Validation by Country

There are not many country tables where the Faroe Islands features in the top 20, but of the 242 tests we ran against clients located in the Faroe Islands we found one quarter of them performed a full DNSSEC validation.

To what extent can these results be attributed to Google’s Public DNS service? Table 4 shows the same 20 countries, but with additional columns added, namely the proportion of use of Google’s Public DNS service, and also looking at just the subset of end clients who are observed to perform DNSSEC validation, and the proportion of these clients who use Google’s Public DNS Service.

Rank

CC

Count

DNSSEC (%)

Google Public DNS (%)

DNSSEC + Google DNS (%)

Country

Valid

Mixed

Not

All

Mixed

None

Count

All

Mixed

None

1

SE

7,082

79.64

4.90

15.46

2.27

1.28

96.44

5640

1.84

0.23

97.93

Sweden

2

SI

5,357

60.35

6.03

33.62

5.30

0.62

94.08

3233

7.42

0.28

92.30

Slovenia

3

AG

313

57.51

8.63

33.87

4.47

2.56

92.97

180

4.44

0.56

95.00

Antigua and Barbuda

4

LU

734

46.59

7.08

46.32

1.77

0.14

98.09

342

1.75

0.00

98.25

Luxembourg

5

AO

247

42.91

21.46

35.63

18.22

6.88

74.90

106

1.89

7.55

90.57

Angola

6

FI

2,748

39.30

16.78

43.92

1.31

0.29

98.40

1080

2.31

0.28

97.41

Finland

7

VN

30,273

38.41

4.06

57.53

39.23

2.93

57.84

11629

96.60

2.30

1.10

Vietnam

8

CZ

33,005

34.97

8.27

56.76

7.02

2.82

90.15

11543

11.87

3.99

84.15

Czech Republic

9

CL

52,924

31.22

8.45

60.33

1.57

1.00

97.43

16524

3.55

0.42

96.03

Chile

10

JM

1,752

30.02

3.20

66.78

28.65

0.63

70.72

526

91.83

0.57

7.60

Jamaica

11

IE

8,985

28.79

5.49

65.72

4.26

1.69

94.05

2587

11.94

1.01

87.05

Ireland

12

NC

290

27.93

5.52

66.55

4.14

1.72

94.14

81

11.11

0.00

88.89

New Caledonia

13

BB

1,476

25.81

1.69

72.49

2.71

0.14

97.15

381

8.14

0.26

91.60

Barbados

14

FO

242

24.79

0.83

74.38

1.65

0.00

98.35

60

3.33

0.00

96.67

Faroe Islands

15

UA

27,934

23.17

12.70

64.13

12.03

3.15

84.82

6473

20.22

2.39

77.38

Ukraine

16

ID

63,288

22.84

8.06

69.10

17.60

5.75

76.65

14454

68.19

12.85

18.96

Indonesia

17

ZA

3,333

22.47

9.72

67.81

3.75

2.16

94.09

749

6.68

1.87

91.46

South Africa

18

US

165,630

19.91

3.59

76.50

3.16

1.10

95.75

32985

7.27

0.75

91.98

United States of America

19

TR

52,909

19.39

2.22

78.39

19.55

1.71

78.74

10260

93.36

3.25

3.39

Turkey

20

AF

232

18.53

34.05

47.41

24.57

27.59

47.84

43

72.09

13.95

13.95

Afghanistan

2,871,753

8.73

6.93

84.34

5.63

3.73

90.64

250635

46.63

6.25

47.12

World

Table 4 – DNSSEC Validation and use of Google’s Public DNS by Country

The relative number of clients who exclusively use Google’s Public DNS service is prominent in Vietnam, Jamaica and Afghanistan, while the relative number of clients who use Google’s DNS service in conjunction with other servers is prominent in Angola, Indonesia and Turkey.

If we look at the subset of clients who are seen to be performing DNSSEC validation, then there is a very strong correlation between the use of Google’s Public DNS resolvers and DNSSEC validation in Vietnam, Jamaica and Indonesia. In other countries in this list it would appear that there are other resolvers used by these clients that also are performing DNSSEC validation.

What we find for the entire data set gathered in May 2013, is that of the 240,635 end clients who are performing DNSSEC validation, some 47% of these clients use Google’s Public DNS service exclusively, another 47% of these clients do not appear to use Google’s Public DNS service, and the remaining 6% use a mix of Google’s and other DNS resolvers.

What this shows is that some 4% of the Internet’s user base exclusively use Google’s Public DNS service and now are having their DNS names validated by this public DNS service. A further 4% of users also use DNS resolvers that perform DNSSEC validation, but do not use Google’s public DNS service to do so.

But doesn’t the final line of table 4 indicate that 5.6% of the tested clients exclusively use Google’s Public DNS? Does this mean that 1.6%, or 45,000 clients, who are exclusively using Google’s Public DNS are not performing DNSSEC validation? It would appear so. Part of the issue with measuring DNS is that an authoritative server cannot unravel the manner by which a DNS query is passed from a client through a DNS forwarder chain before it reaches an authoritative name server. If a DNS Forwarder passes all its queries to Google’s Public DNS service, but marks all its queries with DNSSEC Checking Disabled, then we would see queries from Google’s public DNS resolvers that are not apparently performing DNSSEC validation. This would appear to be the case here, and this is the most likely explanation for these “missing” 45,000 DNSSEC validating clients.

Table 4 displayed a country-by-country view of DNSSEC deployment. What is the view of DNSSEC when looking at service provider networks? This is shown in Table 5.

Rank

CC

Count

DNSSEC (%)

Google Public DNS (%)

DNSSEC + Google DNS (%)

AS Name

Valid

Mixed

Not

All

Mixed

None

Count

All

Mixed

None

1

34630

208

98.08

0.96

0.96

98.08

0.96

0.96

204

99.02

0.98

0.00

AMBRA RO

2

44034

351

97.72

1.14

1.14

1.99

0.28

97.72

343

2.04

0.00

97.96

HI3G Hi3G Access SE

3

197121

478

97.28

0.84

1.88

0.63

0.21

99.16

465

0.65

0.22

99.14

DIODOS Research Net GR

4

12912

1632

97.12

1.78

1.10

2.33

0.12

97.55

1585

2.27

0.13

97.60

Polska Telefonia PL

5

27831

713

96.91

2.95

0.14

1.40

0.42

98.18

691

0.43

0.00

99.57

Colombia Mobil CO

6

44143

408

96.81

2.21

0.98

0.00

0.98

99.02

395

0.00

1.01

98.99

Vip mobile RS

7

5628

530

96.79

1.89

1.32

2.45

0.38

97.17

513

1.56

0.39

98.05

Slovak Telecom SK

8

198471

762

96.59

1.18

2.23

97.38

0.66

1.97

736

99.73

0.27

0.00

Linkem spa IT

9

39651

971

96.50

2.47

1.03

0.82

0.00

99.18

937

0.85

0.00

99.15

Com Hem Sweden SE

10

34779

976

96.41

1.33

2.25

1.43

0.41

98.16

941

1.38

0.32

98.30

T-2 SI

11

34525

299

96.32

1.67

2.01

0.00

0.33

99.67

288

0.00

0.35

99.65

Konrad Baranowski PL

12

44489

482

96.27

1.66

2.07

2.90

1.87

95.23

464

2.37

1.94

95.69

STARNET Starnet CZ

13

52400

315

96.19

1.27

2.54

96.83

0.63

2.54

303

99.67

0.33

0.00

Olo del Peru PE

14

5603

1564

96.10

1.53

2.37

0.64

0.19

99.17

1503

0.60

0.20

99.20

Telekom Slovenije SI

15

27668

525

95.81

2.48

1.71

7.43

0.76

91.81

503

7.36

0.60

92.05

ETAPA EP EC

16

37457

499

94.59

2.20

3.21

1.20

0.20

98.60

472

1.27

0.21

98.52

Telkom-Internet ZA

17

719

706

94.33

2.97

2.69

0.99

0.42

98.58

666

1.05

0.15

98.80

ELISA-AS Elisa Oyj EU

18

29562

852

94.01

3.99

2.00

1.17

0.00

98.83

801

1.12

0.00

98.88

Kabel BW GmbH DE

19

39309

213

92.02

2.82

5.16

1.41

1.41

97.18

196

1.02

1.53

97.45

Edutel B.V. NL

20

6849

5184

91.94

3.09

4.98

3.97

1.99

94.04

4766

3.42

1.68

94.90

UKRTELECOM UA

Table 5 – DNSSEC Validation and use of Google’s Public DNS by Origin AS

Table 5 shows a similar analysis of the DNSSEC data, but this time using the originating network, or Autonomous System, rather than country. Of the top 20 DNSSEC validating networks with more than 200 users seen in this experiment, only three networks, Ambra (AS 34630) in Romania, Linkem (AS 198471) in Italy, and Olo Peru (AS 52400) in Peru, appear to have directed all their customers' DNS queries to Google’s DNS systems. The other 17 networks in this list all appear to be using local DNS resolvers that have been configured to perform DNSSEC validation.

Google’s Public DNS Service

Since March 2013 we’ve seen the proportion of end users who use DNSSEC resolvers that perform DNSSEC validation rise from 3.3% to 8.1%, or a rise of some 4.7%.

Most, but not all of this rise, can be attributed to Google’s Public DNS service, which is used exclusively by some 5.6% of all clients across the entire Internet. When Google turned on DNSSEC validation on their resolvers then the majority of these clients were then performing DNSSEC validation even though they had not changed any part of their local DNS configuration. Just over one half of all clients who are seen to be performing DNSSEC validation, use Google’s Public DNS Servers.

However, that's not the total population of users who avail themselves of Google’s DNS services. A further 3.7% of users also use Google’s Public DNS service, but they use it in conjunction with other DNS resolvers. The most common case here is that the other DNS resolvers used by the client, or by the client’s local DNS resolver, do not use DNSSEC validation, so the SERVFAIL response from the query to Google’s service prompts additional DNS queries to be made to other configured DNS resolvers. A total of 9.4% of the clients we saw in this experiment made use of Google’s Public DNS service in one way or another.

Performing DNS resolution for almost 10% of the Internet is a very significant undertaking by Google. There are some interesting issues this figure raises in terms of the breadth and volume of information about user behaviors that is exposed in DNS queries. In the light of the current concerns over the use of so-called Internet meta-data by a number of national intelligence operations, and broader concerns relating to individual privacy and the use of various forms of cloud services and other public services, this aspect of today’s DNS operations raises some further questions about extent to which certain aspects of user behaviors are visible to this individual DNS operator though the use of their DNS systems by some 10% of the Internet’s user population. However, that is perhaps heading down the path of a new topic, and is straying from this study of the level of deployment of DNSSEC validation in today’s DNS.

On the topic of DNSSEC deployment, we’ll continue to track the state of DNSSEC deployment in the coming months, to see how and where the story of DNSSEC deployment evolves.

Disclaimer

The views expressed are the authors' and not those of APNIC, unless APNIC is specifically identified as the author of the communication. APNIC will not be legally responsible in contract, tort or otherwise for any statement made in this publication.

About the Author

GEOFF HUSTON B.Sc., M.Sc., has been closely involved with the development of the Internet for many years, particularly within Australia, where he was responsible for the initial build of the Internet within the Australian academic and research sector. He is author of a number of Internet-related books, and has been active in the Internet Engineering Task Force for many years.