Post navigation

Faster golang builds

I’ve been hacking in golang since before version 1.4, and the speed at which my builds finished has been mostly trending downwards. Let’s look into the reasons and some fixes. TL;DR click-bait title: “Get 4x faster golang builds with this one trick!”.

There have been slight improvements with newer versions, however Google’s focus has been improving runtime performance instead of build performance. This is understandable since they want to save lots of electricity dollars at scale, which is not as helpful for smaller shops where the developer iteration cycle is the metric to optimize.

This could still be improved if folks wanted to put in the effort. A gcc style -O0 option could help. The sad thing about this whole story is that “instant” builds were a major marketing feature of early golang presentations.

Project size

Over time, my main project (mgmt) has gotten much bigger! It compiles in a number of libraries, including etcd, prometheus, and more! This naturally increases build time and is a mostly unavoidable consequence of building cool things and standing on giants!

This mostly can’t be helped, but it can be mitigated…

Dependency caching

When you build a project and all of its dependencies, the unchanged dependencies shouldn’t need to be rebuilt! Unfortunately golang does a great job of silently rebuilding things unnecessarily. Here’s why…

When you run a build, golang will attempt to re-use any common artefacts from previous builds. If the golang versions or library versions don’t match, these won’t be used, and the compiler will redo this work. Unfortunately, by default, those results won’t be saved, causing you to waste CPU cycles every time you test!

When the intermediate results are kept, they are found in your $GOPATH/pkg/. To save them, you need to either run go install (which makes a mess in your $GOPATH/bin/, or you can run go build -i. (Thanks to Dave for the tip!)

The sad part of this story is that these aren’t cached by default, and stale results aren’t discarded by default! If you’re experiencing slow builds, you should rm -rf $GOPATH/pkg/ and then go build -i. After one successful build, future builds should be much faster!

I don’t like assigning blame, but this feels like a case of the golang tools being obtuse, and the man pages being non-existent. The golang project has a lot of maturing to do to integrate sanely with a stockGNU environment:

Hope this helped improve your golang experience! I always knew something was going in in $GOPATH/pkg/, but I think it’s pretty absurd that I only fully understood it now. My builds are about 4x faster now. :)