Gopher Academy Blog

Community Contributed Go Articles and Tutorials

December 14, 2017

Contributed by

Write a Kubernetes-ready service from zero step-by-step

If you have ever tried Go, you probably know that writing services with Go is an easy thing. Yes, we really need only few lines to be able to run http service. But what do we need to add if we want to prepare our service for production? Let’s discuss it by an example of a service which is ready to be run in Kubernetes.

If we want to try it for the first time, go run main.go might be enough. If we want to see how it works, we may use a simple curl command: curl -i http://127.0.0.1:8000/home. But when we run this application, we see that there is not any information about its state in the terminal.

Step 2. Add a logger

First of all, let’s add a logger to be able to understand what is going on and to be able to log errors and other important situations. In this example we will use the simplest logger from the standard Go library, but for a production-ready service you might be intersted in more complicated solutions such as glog or logrus.

For example, we might want to log 3 situations: when the service is starting, when the service is ready to handle requests and if http.ListenAndServe returns an error. As the result we will have something like this:

Step 3. Add a router

Now, if we write a real application, we might want to add a router to be able to handle different URIs and HTTP methods and match other rules in an easy way. There is not any router in the standard Go library, so let’s use gorilla/mux which is pretty compatible with the standard net/http library.

If your service needs some significant amount of different routing rules, it makes sense to move all routing-related things to separate functions or even a package. Let’s move router initialization and rules to the package handlers (see the full change here).

Let’s add Router function which returns a configured router and home function which handles /home path. Personally, I prefer to use separated files for such things:

Here we check if GET method for /home returns code 200. On the other hand, if we try to send POST we expect 405. And, finally, for a route which does not exists we expect 404. Actually, this example might be a bit “verbose” because the router is already well-tested as a part of gorilla/mux package, so you might want to check even less things.

For home it might make sense to check its response code and response body:

Step 5. Configuration

Next important question is ability to configure our service. Right now it always listens on the port 8000, and probably it might be useful to be able to configure this value. The Twelve-Factor App manifesto, which represents a really great approach for writing services, tells us that it is good to store configuration based on the environment. So, let’s use environment variables for it:

In this example, if the port is not set, the application will simply exit with an error. There is no sense to try continue working, if the configuration is wrong.

Step 6. Makefile

Few days ago there was an article about the make tool, which is very helpful if you want to automate some repeatable routines. Let’s see how we can use it for our application. Currently, we have two actions: to run the tests, to compile and run the service. Let’s add these action to a Makefile. But instead of simple go run we will use go build and we will run a compiled binary, because this approach suits to our production-readiness goals better:

In this example we moved a binary name to a separated variable APP to not to repeat the name few times.

Here, if we want to run an application, we need to delete an old binary (if it exists), to compile the code and to run a new binary with the right environment variable and to do all these things we can use make run.

Step 7. Versioning

The next technique we will add to our service is versioning. Sometimes it might be very useful to know what are the exact build and commit we use in production and when the binary was built.

To be able to store this information let’s add a new package - version:

version/version.go

package version
var (
// BuildTime is a time label of the moment when the binary was built
BuildTime = "unset"
// Commit is a last commit hash at the moment when the binary was built
Commit = "unset"
// Release is a semantic version of current build
Release = "unset"
)

Step 9. Health checks

In a case if we want to run a service in Kubernetes, we usually need to add the health checks: liveness and readiness probes. The purpose of a liveness probe is to understand that the application is running. If the liveness probe fails, the service will be restarted. The purpose of a readiness probe is to understand if the application is ready to serve traffic. If the readiness probe fails, the container will be removed from service load balancers.

To define the readiness probe we usually need to write a simple handler which always return response code 200:

Here we want to mark that the application is ready to serve traffic after 10 seconds. Of course, in the real life there is not any sense to wait for 10 seconds, but you might want to add here cache warming (if your application uses cache) or something like this.

As usual, the whole changes we made of this step you can find on Github.

Note.If your application hits too much traffic, its endpoints will response unstable. E.g. liveness probe might be failed because of timeouts. This is why some engineers prefer to not to use liveness probe at all. Personally, I think that it would be better to scale resources if you find out that you have more and more requests. For example, you might want to scale pods with HPA.

Step 10. Graceful shutdown

When the service needs to be stoped, it is good to not to interrupt connections, requests and other operations immediately, but to handle all those things properly. Go supports graceful shutdown for http.Server since version 1.8. Let’s see how we may use it:

In this example we are able to catch operation system signals and if one of SIGINT or SIGTERM is catched, we will shut down the service gracefully.

Note.When I was writing this code, I tried to catch SIGKILL here. I saw it few times in different libraries and I was sure that it worked. But, as Sandor Szücs noted, it is not possible to catch SIGKILL. In the case of SIGKILL, the application will be stoped immediately.

Step 11. Dockerfile

Our application is almost ready to be run in Kubernetes. Now we need to dockerize it.

The simplest Dockerfile, we need to define here, might look like this:

Dockerfile

FROM scratch
ENV PORT 8000
EXPOSE $PORT
COPY advent /
CMD ["/advent"]

We create the smallest container, copy the binary there and run it (we also do not forget about PORT configuration variable).

Let’s change a bit the Makefile to be able to build an image and run a container. Here it might be useful to define new variables: GOOS and GOARCH which we will use for cross-compilation in the build goal.

We also added the container goal to be able to build an image and the run goal to run our application from the container. All changes are available here.

Now let’s try make run to check the whole process.

Step 12. Vendoring

We have an external dependency (github.com/gorilla/mux) in our project. And it means that for production readiness we definetely need to add dependency management here. If we use dep the only thing which we need for our service is dep init:

The CONTAINER_IMAGE variable defines a Docker registry repo which we will use to push and pull our service images. As you can see, in this case it includes the username (webdeva). If you do not have an account at hub.docker.com yet, please create it and login with docker login command. After this, you will be able to push images.

Let’s define the necessary Kubernetes configuration (manifest). Usually, for the simplest service we need to set at least deployment, service and ingress configurations. By default the manifests are static, it means that you are not able to use any variables there. Hopefully, you can use helm to be able to create flexible configuration.

In this example we will not use helm, but it might be useful to define a couple of variables: ServiceName and Release, it gives us more flexibility. Later, we will use the sed command to be able to replace these “variable” with the real values.

It is better to discuss Kubernetes configuration as a part a separate article, but, as you can see, here, among other things, we defined where it is possible to find a container image and how to reach liveness and readiness probes.

These commands “compile” all *.yaml configurations to a single file, replace Release and ServiceName “variables” by the real values (please, note that here I use gsed instead of the standard sed) and run kubectl apply to install the application to Kubernetes.

It might be interesting for you how a more flexible service, prepared for the real production, may look like. In this case, feel free to take a look at takama/k8sapp, a Go application template which meets the Kubernetes requirements.