Shared infrastructure as code

If you follow DevOps
trends, you have likely
heard of infrastructure as
code. Tangibly, infrastructure as code means having things like your network configuration, server attributes, access control, etc. in a machine-readable format. This code then:

Serves as the source of truth for what the infrastructure should look like

Can be used to recreate the infrastructure from scratch

Is under version control

Is modified by pull requests

DNS is a major
piece of shared infrastructure at the Technology Transformation Services
(TTS, which 18F is part of), which made it a prime candidate for
infrastructure as code. Of the many benefits, doing DNS changes
through pull requests rather than tickets brought turnaround time down from multiple days to under ten minutes.

What’s DNS?

TTS manages DNS for a number of domains. When you visit somewebsite.gov,
DNS tells your browser which server to query for the website. Think of
DNS like a phone book for the internet, mapping names to numbers (in
this case, hostnames to IP addresses). As an example, the
output below shows that 18f.gsa.gov points to a CloudFront CDN domain, which
in turn points to a handful of IP addresses.

While this is mostly checking for simple things like valid syntax right now, we have ideas for expanding, like reporting unused
records.

Before merge, someone looks over every change, even if it’s just a
sanity check. Merging a pull request triggers an automatic deploy of the new DNS settings to Amazon Route53, which is where the live DNS records (name servers) reside. While this process is like submitting a ticket, administrators don’t need to manually enter any changes.

With the Terraform code in Git, we get all the benefits of version
control, particularly an auditable history, attribution, and the ability to “undo”. (“Who changed this record? When? Why? That broke
things…let’s take it back to how it was before.”) If our live records get deleted somehow, the next deploy puts them back in place exactly as they were.

Migration

Earlier this year, we needed to move our DNS records to a new Amazon Web Services account. Manually copying and verifying these records by hand would have been a tedious process. On top of that, we had to maintain all records in both places during the migration.

With the configuration in code and version control, we were able to
modify our continuous
deployment so that every change
to the records was deployed to the old and new accounts.

The
declarative
nature of Terraform means that you tell it what state you want the
infrastructure to be in, and it figures out what changes to make to get there. On its first run against the new account, Terraform saw that none of the records were in place, so it created them. Time from zero to hundreds of records present in the new account? A half hour.

What could have been a painful DNS migration was made calm and
confident by infrastructure as code and continuous deployment. Better Living Through Automation!

Do it yourself

Having TTS’ DNS specified in code in a public repository with continuous integration and deployment has been unequivocally positive. If your organization doesn’t do infrastructure as code but is interested in trying it out, DNS is a great place to start. Public DNS records don’t contain any sensitive
information, it’s
easy to confirm that they are deployed correctly, and you can use
basically any continuous integration and DNS providers (that have an
API). Fork our DNS repository to get
started!

Related posts

Often when developing open source software, and especially software that relies on outside services, you’ll find that you have to manage sensitive information. While there are a large number of things that can be considered sensitive, open source developers often deal with sensitive items such as API tokens, passwords, and private keys that are required for the system to function. Here's how we approached keeping this information safe.

Security compliance is a major factor in launching a software system in the federal government. The Authority To Operate compliance process for systems within our division of GSA was taking more than six months for every system. With the new process, we have cleared the backlog and reduced the turnaround time to under a month.

The National Day of Civic Hacking is a great time to attend — or host your first hackathon. We’ve outlined some strategies for first-time hackathon hosts to ensure new participants feel welcome, and help them maximize their contributions.