Please follow the below template, it will help us to help you!

Expected Behaviour:

PiHole should be resolving DNS addresses correctly when using DNSSEC

Actual Behaviour:

Since the update to the FTLDNS Branch, I have experienced an increase in the number of Bogus results for DNSSEC resolution. This means that the page fails to load correctly. I feel like this is due to a misconfiguration in pihole FTLDNS, but can’t be sure.

I have PiHole setup to use the Stubby daemon running on a local interface to resolve DNS-over-TLS from the Cloudflare 1.1.1.1 servers. Stubby is set up with DNSSEC. Supposedly Stubby doesn’t need a trust anchor (the option for “configuration free DNSSEC” is selected in Stubby config).

Perhaps this is the issue, but I am sceptical of that, as it was working fine with very few Bogus results. Now, after an update to FTLDNS, Bogus results have increased.

Debug Token:

[Replace this text with the debug token provided from running pihole -d (or running the debug script through the web interface]

Did you remove dnsmasq when you installed FTLDNS? If not and it’s still installed, can you tell us the version that was installed? (If you’re on a Debian based distro then apt cache policy dnsmasq should give you that information.) Thanks.

Also, forgot to add something that is perhaps relevant. The issue is very intermittent. One minute the resolution will fail, and in the PiHole web-interface query log it will say BOGUS under DNSSEC status. The next minute, after refreshing the page/trying again, the resolution will succeed, and the DNSSEC status will say INSECURE (the standard response for a domain that is not setup to use DNSSEC)

Since the update to the FTLDNS Branch, I have experienced an increase in the number of Bogus results for DNSSEC resolution.

gecko:

Installed: 2.76-5+rpt1+deb9u1

One major difference with FTLDNS is that you’re now running dnsmasq 2.79 under the hood. From the official CHANGELOG, I see quite a few changes concerning DNSSEC (only quoting DNSSEC relevant changes):

version 2.79
Add support for Ed25519 signatures in DNSSEC validation.
No longer support RSA/MD5 signatures in DNSSEC validation,
since these are not secure. This behaviour is mandated in
RFC-6944.
Use SIGINT (instead of overloading SIGHUP) to turn on DNSSEC
time validation when --dnssec-no-timecheck is in use.
Note that this is an incompatible change from earlier releases.
Fix for DNSSEC with wildcard-derived NSEC records.
It's OK for NSEC records to be expanded from wildcards,
but in that case, the proof of non-existence is only valid
starting at the wildcard name, *.<domain> NOT the name expanded
from the wildcard. Without this check it's possible for an
attacker to craft an NSEC which wrongly proves non-existence.
Thanks to Ralph Dolmans for finding this, and co-ordinating
the vulnerability tracking and fix release.
CVE-2017-15107 applies.
version 2.77
Fix problem with --dnssec-timestamp whereby receipt
of SIGHUP would erroneously engage timestamp checking.
Thanks to Kevin Darbyshire-Bryant for this work.
Fix foobar in rrfilter code, that could cause malformed
replies, especially when DNSSEC validation on, and
the upstream server returns answer with the RRs in a
particular order. The only DNS server known to tickle
this is Nominum's. Thanks to Dave Täht for spotting the
bug and assisting in the fix.

I don’t think anything of this should really be affecting you, but you never know…

gecko:

Stubby is set up with DNSSEC.

Do you have any logs with stubby? Can you see in these logs if it told our resolver that the DNSSEC status was INSECURE but FTLDNS said BOGUS instead? Can you quote the relevant log lines from /var/log/pihole.log?

gecko:

I feel like this is due to a misconfiguration in pihole FTLDNS, but can’t be sure.

Without wanting to reject possible blame, I doubt that this can be the case. Our resolver uses the exact same configuration, your dnsmasq used before (we do not imply any configuration except from what is set up within /etc/dnsmasq.d/).

gecko:

The next minute, after refreshing the page/trying again, the resolution will succeed, and the DNSSEC status will say INSECURE

As I said above it would be very helpful to get some more logs.

Also - just for testing! - would you be willing to directly use an upstream DNS server for testing? By this, we could exclude that the problem is coming from stubby or its configuration.

I’ve been mailing with Simon Kelley (developer of dnsmasq) for a while now, regarding this DNSSEC problem.
I’m NOT using FTLDNS, but regularly upgade dnsmasq, using the test releases (currently running dnsmasq2.80test2)
It appears this problem appeared in dnsmasq2.77 and is still there in dnsmasq2.79.

dnsmasq2.80test1 & dnsmasq2.80test2 attempt to fix this problem, however, they make the problem even more visible, so for now, I’m running dnsmasq with the setting ‘dnssec-check-unsigned=no’. I’m not sure this setting even works in earlier versions…

from the developer, regarding this setting:
‘quote’
However, this is dangerous, since it allows dnsmasq to accept forged answers even to queries which should be signed.
‘/quote’

Yeah, I worked out it probably has nothing to do with stubby. I found out yesterday from the stubby github that I shouldn’t even have DNSSEC enabled in the stubby config if I am only using stubby to forward onto dnsmasq, as it just increases overhead needlessly (I assumed DNSSEC had to be enabled in both pihole/dnsmasq and stubby for it to work).

I disabled DNSSEC in stubby, and DNSSEC still works correctly in piholle (raspberrypi.org comes back as secure, for example). But the problem with BOGUS results is still there. So many sites are coming back as BOGUS and faiul to resolve that I think I may have to disable DNSSEC for now, as it’s really interrupting my workflow having to keep waiting a few mins then refreshing pages for them to resolve.

The reason I felt it was an issue in FTLDNS is that I’ve had this configuration for a few weeks (since cloudflare 1.1.1.1 was made available on the first), but it’s only in the last week that I’ve had these issues, and since pihole FTLDNS branch is the only thing that’s been updated in that time, I assumed it was that.

I checked the stubby logs, and there was nothing in there I could see regarding DNSSEC queries. Like I said before, I was told by people on the stubby github that I should disable DNSSEC on stubby anyway, if I am forwarding requests on to dnsmasq/FTLDNS.

An example in the pihole log is from this morning. I was having an issue downloading an update on the Apple AppStore. This was in my log

A part of the problem has been identified.
As mentioned earlier, I’m running dnsmasq2.80test2 and dnscrypt-proxy 2.0.11

I only have ipv4, my provider refuses to exchange my docsis 2 (ipv4 only) modem for a docsis 3 modem, so …

In the wiki, section ‘making things go faster’, there is a setting ‘block_ipv6 = false’. The wiki says to change it, so I did.
WRONG!!!If you only have IPv4 and you’re using dnsmasq + dnscrypt-proxy V2 + DNSSEC, don’t change that setting!!!

justification (from the dnsmasq developer):
‘quote’
Looking at your logs, this is responsible for some of the BOGUS validations, if dnscrypt-proxy gives a synthetic reply to a query in a signed domain, that will fail DNSSEC validation, obviously.
If your network doesn’t support IPv6, chances are that your applications are still constantly trying to resolve IPv6 addresses, causing unnecessary slowdowns.
This causes the proxy to immediately reply to IPv6 requests, without having to send a useless request to upstream resolvers, and having to wait for a response.
This uses a plugin that requires dnscrypt-proxy to be compiled with the ldns library.
‘/quote’

I changed the setting back to it’s default (‘block_ipv6 = false’) and some of the BOGUS errors have disappeared. In the item above (something strange I noticed), ‘archive.raspberry.org’ came up as BOGUS, when using ‘sudo apt-get update’.
That problem is now solved by reverting to the default dnscrypt-proxy setting, however, still a long way to go for other false BOGUS errors…

DNSSEC problems are something we can really only reduce by always offering the latest (stable) version of dnsmasq through FTLDNS. It is clear to us that the DNSSEC implementation is dnsmasq is far from being completely bug free. This is also the reason why we never recommend to enable DNSSEC for Pi-hole’s in any productivity level installation of Pi-hole as you have to expect catches and non-resolving domains.

I know that this is not particularly helpful, but there are few things we really can do about it. Furthermore, I’m not convinced that DNSSEC is really “worth it” (at least not right now) as even the majority of the big global players aren’t investing in this (they always come as INSECURE).

The reason I felt it was an issue in FTLDNS is that I’ve had this configuration for a few weeks (since cloudflare 1.1.1.1 was made available on the first), but it’s only in the last week that I’ve had these issues, and since pihole FTLDNS branch is the only thing that’s been updated in that time, I assumed it was that.

I can understand this, but I can also assure you that with respect to DNSSEC, nothing has changed since the very early days of FTLDNS.

I move this from the Help category over to General as there isn’t much we can do about it right now except from updating FTLDNS’s internal resolver to the next dnsmasq version once this is considered stable.

Furthermore, I’m not convinced that DNSSEC is really “worth it” (at least not right now) as even the majority of the big global players aren’t investing in this (they always come as INSECURE).

DL6ER, nothing personal, absolutely NOT, apologies if I offend you.
But this is a bad statement that doesn’t help furthering DNSSEC.
Tech people should grab every opportunity to request (tech) sites to create DNSSEC records.discourse.pi-hole.net should of had DNSSEC records the day pihole first supported it.

I’m not convinced that DNSSEC is really “worth it” (at least not right now)

(ignore the part with the global players for now). I didn’t intend to say DNSSEC will never worth it, but, in fact, it isn’t today. Users enable DNSSEC and start experiencing problems. If they are not able to properly analyze the root of the issues, the only thing they get is a screwed up Internet experience.

This statement is a pure support-oriented statement (talking to users on any level of technical experience and knowledge of DNS). I, in fact, run a recursive DNS server on the same device Pi-hole is running on (I think you are aware of the wiki article I wrote). Here, I fully use DNSSEC (I even use harden-dnssec-stripped), but then I’ also aware of what I need to do in case some domains don’t resolve erroneously. I just cannot be bothered to recommend switching on DNSSEC right now for an arbitrary user (the benefit is low as long as many domains are anyhow INSECURE).

Oh, we actually had DNSSEC for some time, but it was causing issues all over the place. When you are INSECURE, then users can visit your page just fine. However, as soon as you configure DNSSEC for your page, each very minor perturbation will immediately render your page BOGUS and users with DNSSEC validation won’t be able to use Discourse any more.

jpgpi250:

If the DNS administrators keep being annoyed with requests to support DNSSEC, sooner or later at least some of them will create them…

If it would only be so simple … we tried hard for some time and eventually came to the conclusion that it is not worth the trouble it was causing. @DanSchaper would be able to give you more details on our implementation efforts on the Pi-hole domains and why it got removed again.

We did have DNSSEC enabled, but rolling DS keys to the registrars that are not the same as our DNS providers is not the most elegant solution. I already spend more than full time hours supporting the infrastructure that makes everything work, and I don’t have time to shuffle records between providers, between registrars and have to worry that all of our TLDs will be non-resolvable if something goes wrong somewhere, either with my process or with one of our providers. Maybe when the process matures it will be revisited, but for now it’s too much work, too prone to failures and too manually intensive to implement.

Also, I should mention that many registrars and DNS providers do not even offer the option of DNSSEC because of the high support load it creates. It’s not like a bad record can time out with a cache purge or a TTL time out, you break the chain and it’s game over for a long time. We can’t risk that.

I’m not sure the best solution is to simply disable DNSSEC. I realise it comes with its technical difficulties and bugs, but at the moment it’s the best we’ve got for some kind of DNS security against DNS hijacking (along with DNSCrypt/DNS-over-TLS/DoH), and until the industry comes up with something better, we should be encouraged to use it. Only with more people using it can we hope for it to either be improved, or a better replacement for it to be made.

I had a thought though. If it is, as suspected, due to issues between dnsmasq and DNSSEC, would it better for me to disable DNSSEC in the pihole interface (and thus on the FTLDNS/dnsmasq side of things), and only enable it on the stubby resolver? Of course, I wouldn’t then be able to tell from the pihole UI what the issue was if there was a problem with DNSSEC resolution, but I assume that any result that was genuinely BOGUS wouldn’t resolve on stubby, and so wouldn’t be passed on to pihole, and thus I gain the same protection. Or would that not work?

would it better for me to disable DNSSEC in the pihole interface (and thus on the FTLDNS/dnsmasq side of things), and only enable it on the stubby resolver?

Yes, see also this comment I made:

DL6ER:

I, in fact, run a recursive DNS server on the same device Pi-hole is running on (I think you are aware of the wiki article I wrote). Here, I fully use DNSSEC (I even use harden-dnssec-stripped)

I don’t know anything about the stubby resolver, however, if it works as we’d expect, then it wouldn’t give you a reply for a BOGUS domain and hence Pi-hole wouldn’t be able to give your clients a reply and everything would be fine.