Go with Peter Bourgon

Go is meant to be simple, but sometimes the conventions can be a little hard to grasp. I’d like to show you how I start all of my Go projects, and how to use Go’s idioms. Let’s build a backend service for a web app.

Once installed, the only other thing to do is to set your GOPATH. This is the root directory that will hold all of your Go code and built artifacts. The Go tooling will create 3 subdirectories in your GOPATH: bin, pkg, and src. Some people set it to something like $HOME/go, but I prefer plain $HOME. Make sure it gets exported to your environment. If you use bash, something like this should work:

There are a lot of editors and plugins available for Go. I’m personally a huge fan of Sublime Text and the excellent GoSublime plugin. But the language is straightforward enough, especially for a small project, that a plain text editor is more than sufficient. I work with professional, full-time Go developers who still use vanilla vim, without even syntax highlighting. You definitely don’t need more than that to get started. As always, simplicity is king.

A new project

With a functioning environment, we’ll make a new directory for the project. The Go toolchain expects all source code to exist within $GOPATH/src, so we always work there. The toolchain can also directly import and interact with projects hosted on sites like GitHub or Bitbucket, assuming they live in the right place.

For this example, create a new, empty repository on GitHub. I’ll assume it’s called “hello”. Then, make a home for it in your $GOPATH.

Invoke go build to compile everything in the current directory. It’ll produce a binary with the same name as the directory.

$ go build
$ ./hello
hello!

Easy! Even after several years of writing Go, I still start all of my new projects like this. An empty git repo, a main.go, and a little bit of typing.

Since we took care to follow the common conventions, your application is automatically go get-able. If you commit and push this single file to GitHub, anyone with a working Go installation should be able to do this:

$ go get github.com/your-username/hello
$ $GOPATH/bin/hello
hello!

Making a web server

Let’s turn our hello, world into a web server. Here’s the full program.

The function hello is an http.HandlerFunc, which means it has a specific type signature, and can be passed as an argument to HandleFunc. Every time a new request comes into the HTTP server matching the root path, the server will spawn a new goroutine executing the hello function. And the hello function simply uses the http.ResponseWriter to write a response to the client. Since http.ResponseWriter.Write takes the more general []byte, or byte-slice, as a parameter, we do a simple type conversion of our string.

Finally, we start the HTTP server on port 8080 and with the default ServeMux via http.ListenAndServe. That’s a synchronous, or blocking, call, which will keep the program alive until interrupted. Compile and run just as before.

$ go build
$ ./hello

And in another terminal, or your browser, make an HTTP request.

$ curl http://localhost:8080
hello!

Easy! No frameworks to install, no dependencies to download, no project skeletons to create. Even the binary itself is native code, statically linked, with no runtime dependencies. Plus, the standard library’s HTTP server is production-grade, with defenses against common attacks. It can serve requests directly from the live internet—no intermediary required.

Go is a statically-typed language, so we should create a structure that mirrors this response format. We don’t need to capture every piece of information, just the stuff we care about. For now, let’s just get the city name and temperature, which is (hilariously) returned in Kelvin. We’ll define a struct to represent the data we need returned by the weather API.

The type keyword defines a new type, which we call weatherData, and declare as a struct. Each field in the struct has a name (e.g. Name, Main), a type (string, another anonymous struct), and what’s known as a tag. Tags are like metadata, and allow us to use the encoding/json package to directly unmarshal the API’s response into our struct. It’s a bit more typing compared to dynamic languages like Python or Ruby, but it gets us the highly desirable property of type safety. For more about JSON and Go, see this blog post, or this example code.

We’ve defined the structure, and now we need to define a way to populate it. Let’s write a function to do that.

The function takes a string representing the city, and returns a weatherData struct and an error. This is the fundamental error-handling idiom in Go. Functions encode behavior, and behaviors typically can fail. For us, the GET request against OpenWeatherMap can fail for any number of reasons, and the data returned might not be what we expect. In either case, we return a non-nil error to the client, who’s expected to deal it in a way that makes sense in the calling context.

If the http.Get succeeds, we defer a call to close the response body, which will execute when we leave the function scope (when we return from the query function) and is an elegant form of resource management. Meanwhile, we allocate a weatherData struct, and use a json.Decoder to unmarshal from the response body directly into our struct.

As an aside, the json.NewDecoder leverages an elegant feature of Go, which are interfaces. The Decoder doesn’t take a concrete HTTP response body; rather, it takes an io.Reader interface, which the http.Response.Body happens to satisfy. The Decoder supplies a behavior (Decode) which works just by invoking methods on types that satisfy other behaviors (Read). In Go, we tend to implement behavior in terms of functions operating on interfaces. It gives us a clean separation of data and control planes, easy testability with mocks, and code that’s a lot easier to reason about.

Finally, if the decode succeeds, we return the weatherData to the caller, with a nil error to indicate success. Now let’s wire that function up to a request handler.

Here, we’re definining the handler inline, rather than as a separate function. We use strings.SplitN to take everything in the path after /weather/ and treat it as the city. We make our query, and if there’s an error, we report it to the client with the http.Error helper function. We need to return at that point, so the HTTP request is completed. Otherwise, we tell our client that we’re going to send them JSON data, and use json.NewEncoder to JSON-encode the weatherData directly.

The code so far is nice and procedural, and easy to understand. No opportunity for misinterpretation, and no way to miss the common errors. If we move the “hello, world” handler to /hello, and make the necessary imports, we have our complete program:

Querying multiple APIs

Maybe we can build a more accurate temperature for a city, by querying and averaging multiple weather APIs. Unfortunately for us, most weather APIs require authentication. So, get yourself an API key for Weather Underground.

Since we want the same behavior from all of our weather APIs, it makes sense to encode that behavior as an interface.

Now, we can transform our old OpenWeatherMap query function into a type that satisfies the weatherProvider interface. Since we don’t need to store any state to make the HTTP GET, we’ll just use an empty struct. And we’ll add a simple line of logging, so we can see what’s happening.

Since we only want to extract the Kelvin temperature from the response, we can define the response struct inline. Otherwise, it’s pretty much the same as the query function, just defined as a method on an openWeatherMap struct. That way, we can use an instance of openWeatherMap as a weatherProvider.

Let’s do the same for the Weather Underground. The only difference is we need to provide an API key. We’ll store the key in the struct, and use it in the method. It will be a very similar function.

(Note that the Weather Underground doesn’t disambiguate cities quite as nicely as OpenWeatherMap. We’re skipping some important logic to handle ambiguous city names for the purposes of the example.)

Notice that the function definition is very close to the weatherProvider temperature method. If we collect the individual weatherProviders into a type, and define the temperature method on that type, we can implement a meta-weatherProvider, comprised of other weatherProviders.

Make it concurrent

Right now we just query the APIs synchronously, one after the other. But there’s no reason we couldn’t query them at the same time. That should decrease our response times.

To do that, we leverage Go’s concurrency primitives: goroutines and channels. We’ll spawn each API query in its own goroutine, which will run concurrently. We’ll collect the responses in a single channel, and perform the average calculation when everything is finished.

Now, our requests take as long as the slowest individual weatherProvider. And we only needed to change the behavior of the multiWeatherProvider, which, notably, still satisfies the simple, synchronous weatherProvider interface.

Commit and push!

Simplicity

We’ve gone from ‘hello world’ to a concurrent, REST-ish backend server in a handful of steps and using only the Go standard library. Our code can be fetched and deployed on nearly any server architecture. The resulting binary is self-contained and fast. And, most importantly, the code is straightforward to read and reason about. It can easily be maintained and extended, as necessary. I believe all of these properties are a function of Go’s steady and philosophic devotion to simplicity. As Rob “Commander” Pike puts it, less is exponentially more.

Further exercises

Can you add another weatherProvider? (Hint: forecast.io is a good one.)

Can you implement a timeout in the multiWeatherProvider? (Hint: look at time.After.)

Peter Bourgon

Go

Go evangelist and SoundCloud engineer

Peter is a software engineer focusing on large-scale distributed systems. He's been writing Go since its public release in 2009, and is a huge fan of its simplicity and pragmatism. Peter lives in Berlin and works at SoundCloud, building the data systems and infrastructure that power the world's leading audio platform.