The Situation

The Problem With The Situation

Build time. BUILD. TIME. Now if you want to run the tests, you have
to gather all of your dependencies, which is fine, because you want
the dependencies to be there when you run the tests.

But what happens when you want to build the thing to deploy it, say
your tests pass? You gotta wait for those dependencies to be gathered
AGAIN. This is long. I am of the Internet Generation, I AM EASILY
BORED.

The Solution

Well. I mean. “A solution”. “My solution”. I humbly propose to you
the following: Multi-stage builds.

NOOOOOOOO

Yesssss.

WHYYYYYYY

Look, it’s gonna hurt less, in the end, I promise.

Let’s get this show on the road:

FROM golang:1.11.1-alpine3.8 AS base
ENV CGO_ENABLED 0
# Install tools required for project
# Run `docker build --no-cache .` to update dependencies
RUN apk update && apk add --no-cache git make protobuf-dev
RUN go get github.com/golang/dep/cmd/dep
FROM base AS deps
# Now you go get all the project's dependencies.
# When using modules, you can simplify all this.
COPY . $GOPATH/src/path/to/your-project
WORKDIR $GOPATH/src/path/to/your-project
RUN dep ensure -vendor-only -v
## Run some tests
FROM deps AS tests
RUN go vet ./...
RUN go test
## Build our service
FROM deps AS build
WORKDIR $GOPATH/src/path/to/your-project
RUN go build -o /service
FROM scratch AS out
COPY --from=build /service /
CMD ["service"]

Separating it in three build stages is important as you’ll see.

What, that’s it?

Hang on, little tomato. You still need to build those.

Say you want to run the tests first, and then only build and push
the final image if those go through?

The above is quite basic, but you can go from there and build it
out however you want. Doing so will use the Docker cache locally,
which means the second docker call will actually build the image
out starting from the deps intermediary image. This can drastically
improve your CI build time.

A Note About CI

In some cases, it might be wise to distribute builds across a cluster of build machines. Were you to do that, I’d recommend tagging intermediary images and pushing them to a private repository; the obvious caveat is that intermediary images can become quite fat, especially if you have build-time dependencies added in the container. For instance, one of my base images easily became 350+ mb. It might be feasible to push out the image from your CI server to your registry, network-wise, but it’s still a nonzero cost, so probably take that into account.

My preferred approach is to try and benefit from the local Docker cache and GNU Parallel so that the jobs that use the same base image (deps in our example) are built concurrently, and make Parallel exit with a non-zero status code if any of the commands fail by running parallel with --halt now,fail=1.

Hopefully reading this was at least a fraction useful to you as it was for me writing it.