Custom Go module URLs with serverless AWS and Terraform

Feb 14, 2019

Go, as a language and as an ecosystem, is known for its minimalism. One example of this is the package management system in the go get command. Import paths are as simple as the URL to the source code repository that holds the package’s code.

This is nice because there is no need for another centrally managed package server (a la RubyGems or NPM) to host distributable packages distinct from source code, but has some limitations compared to central repository model. One limitation is the direct coupling between all import statements that depend on a certain package, and the location where that package’s author has decided to host its code.

Go’s core team introduced a solution to this problem: the ability to mask the import path of packages with so-called “vanity URLs”. For example, I could have my code hosted at github.com/example/my-go-module, but everyone who wanted to use my package could go get go.example.com/my-go-module. That way, if I ever decided to move to BitBucket or GitLab, I could do so without breaking the builds of everyone who depends on my package.

This works by setting up some sort of web server to respond to GET requests to the vanity URL and respond with a custom HTML <meta> tag, pointing clients (go get) to the current source code repository. This behavior is documented here. Google Cloud Platform released a HTTP server that you can run yourself to translate go get requests for you, but personally I’d rather not have to manage another server in my infrastructure, so I decided to build a serverless request translator using AWS’s serverless offerings. Going serverless truly allows you to “set and forget” production systems like nothing else.

Serverless Go Vanity URLs on AWS

These are the components that make up this system.

S3 bucket, as a simple web host to serve up <meta> tags in response to package requests.

CloudFront distribution, to sit in front of the S3 bucket to serve requests via HTTPS (which go get expects).

A free AWS Certificate Manager TLS certificate which we can easily attach to our CloudFront distribution to support HTTPs traffic.

This certificate must be set up in the us-east-1 region because of limitations within CloudFront. I address this in more detail further down the page.

Two Route53 DNS records to

Point go.example.com to the CloudFront distribution.

Validate ownership of this domain to allow AWS Certificate Manager to issue a certificate.

You must define the variables defined at the top of the following Terraform config.

Overall, the Terraform code should be plug-and-play. I didn’t include the Terraform state backend and provider configuration, which I assume almost everybody has. If you don’t have that set up, I recommend following the Terraform “Getting Started” tutorial.

Certificate Setup

One important thing to note is that the AWS Certificate Manager certificate must be set up in the us-east-1 (N. Virginia) region for it to be compatible with CloudFront distributions. If the rest of your infrastructure is in a different region, don’t fret. Terraform handles multi-region setups very nicely, so all you need to do is configure a new provider named aws with the us-east-1 alias, and use that to create just the TLS cert. It will link up with the rest of your resources in your primary region very easily.