Those who can make you believe absurdities can make you commit atrocities. —Voltaire

Main menu

Post navigation

EC2, BIND9, DNS, Pancakes and You!

I finally reached that place where maintaining versions of host files just didn’t seem to cut it andfound myself mucking about with BIND9 to solve the problem of keeping track of all those internal IP addresses when I bring instances online and take them offline. This seemed to be the quickest and easiest way to handle fairly basic networking needs though it does introduce yet another point of failure so I’ll need to tackle primary and secondary DNS servers in the near future and maybe develop a doomsday host file backup but if the latter is invoked I’m sure that I have bigger problems on my hands.

A caveat before we dive in, more to remind me when I have to revisit this some 18 months later. A couple of iterations in I thought it would be cute to set up the DNS zone for the full domain (*.awesome.com) but that seemingly added to my headaches as I tried to examine resources that did not have a DNS entry just yet. Since this is for internal use only we can pretty much set the FQDN to be whatever we want it to be, in this case pancake.man is our working domain.

EDIT: A couple of things occurred to me while I mulled this over. Keeping the zone files in /etc/bind while sounding like a neat idea doesn’t take into account the ephemeral nature of instances so I moved ours into an EBS volume. Additionally, I neglected to add that if your distro uses app armor you’ll have to grant rw permissions to the folder where you are keeping the db files. That can be done by editing /etc/apparmor.d/usr.sbin.named and adding the following:
/some/where/other/than/etc/bind/** rw,

The you can just reload apparmor and be on your happy way.

Generate a key pair….
This is really a CYA implementation but it should ensure that only the servers you want updating the LAN DNS can do it.
dnssec-keygen -a HMAC-MD5 -b 512 -n USER pancake.man

…and copy the secret from the public key as you’ll need it in the next section.

The above block sets up the key that you created earlier and it will be used in the following block that defines the zone.
zone "pancake.man"
{
type master;
file "/etc/bind/zone/db.pancake.man";
allow-update { key pancake.man.; };
allow-query { any; };
};

One thing that tripped me up for a little bit was the formation of the SOA line, pancake.man IN SOA ns.pancake.man. me.pancake.man.. I had been reading ns.pancake.man. me.pancake.man. as FQDN when in fact they are a FQDN and and email address where the ‘@’ has been replaced with a ‘.’; file this one under RIF: Reading Is Fundamental.

Now lets punch an entry in using our shiny set of keys. I found making a script was easier for testing purposes, create a file called dnsupdate.sh and add the following:
#!/bin/bash
cat << EOF | /usr/bin/nsupdate -k Kpancake.man.+157+46088.private -v
server 127.0.0.1
zone pancake.man
update delete $1 A
update add $1 60 A $2
show
send
EOF

The private key is the one we made way back in the beginning of this exercise, your's might be about waffles. After making the file executable, just ./dnsupdate test.pancake.man 10.1.2.3. Ideally, you should get something like the following back:
Outgoing update query:
;; ->>HEADER<
;; flags: ; ZONE: 0, PREREQ: 0, UPDATE: 0, ADDITIONAL: 0
;; ZONE SECTION:
;pancake.man. IN SOA

;; UPDATE SECTION:
test.pancake.man. 0 ANY A
test.pancake.man. 60 IN A 10.1.2.3

Implement your shiny new DNS server
With all that done now we and testing showing that everything works as expected we want to share it with the whole LAN but there's a potential snag. When you work up the next morning still heady from your success you noticed that when DHCP decided to check on its lease it wrote over /etc/resolv.conf with the stock EC2 values, showing absolutely no love for your handiwork. Not what we wanted to happen but there are a couple of steps we can take to make sure that your shiny new DNS persists (this is mainly geared toward Ubuntu so your mileage may vary). The AWS forums has a great writeup about this though I'm only implementing some what was discussed.

Since there is the chance of a DNS server changing, instance goes down or a new one is swapped in, I'm going the route of appending the DNS server to /etc/hosts at boot time and then adding the FQDN to /etc/dhcp3/dhclient.conf like so...
prepend domain-name-servers ns.pancake.man;

What that will do is punch in the IP address specified in /etc/hosts into /etc/resolv.conf when DHCP does it's little dance.

Loose ends...
While everything should be working decently on a small scale there are still plenty of things still left to do like setting up a secondary DNS server for failover, implementing EC2 internal zones for forwarding as described in the AWS post, and working out an elegant solution for updating clients about the DNS server. As for the latter, I'm still playing with scripts to fetch the local IP address of the DNS server, from using an empty security group as a tag to just keeping an updated list in S3 that is maintained by the DNS servers themselves. That, among the other things, will be a post for another day when I finally stumble on something that isn't so ham-fisted.