Go Programming Language: An Introductory Tutorial

What’s the Go Programming Language?

Go is a recent language which sits neatly in the middle of the landscape, providing lots of good features and deliberately omitting many bad ones. It compiles fast, runs fast-ish, includes a runtime and garbage collection, has a simple static type system and dynamic interfaces, and an excellent standard library.

Go and OOP

OOP is one of those features that Go deliberately omits. It has no subclassing, and so there are no inheritance diamonds or super calls or virtual methods to trip you up. Still, many of the useful parts of OOP are available in other ways.

*Mixins* are available by embedding structs anonymously, allowing their methods to be called directly on the containing struct (see embedding). Promoting methods in this way is called *forwarding*, and it’s not the same as subclassing: the method will still be invoked on the inner, embedded struct.

Embedding also doesn’t imply polymorphism. While A may have a B, that doesn’t mean it is a B — functions which take a B won’t take an A instead. For that, we need interfaces, which we’ll encounter briefly later.

Meanwhile, Go takes a strong position on features that can lead to confusion and bugs. It omits OOP idioms such as inheritance and polymorphism, in favor of composition and simple interfaces. It downplays exception handling in favour of explicit errors in return values. There is exactly one correct way to lay out Go code, enforced by the gofmt tool. And so on.

Go is also a great language for writing concurrent programs: programs with many independently running parts. An obvious example is a webserver: Every request runs separately, but requests often need to share resources such as sessions, caches, or notification queues. This means skilled Go programmers need to deal with concurrent access to those resources.

While the Go language has an excellent set of low-level features for handling concurrency, using them directly can become complicated. In many cases, a handful of reusable abstractions over those low-level mechanisms makes life much easier.

In today’s Go programming tutorial, we’re going to look at one such abstraction: A wrapper which can turn any data structure into a transactional service. We’ll use a Fund type as an example – a simple store for our startup’s remaining funding, where we can check the balance and make withdrawals.

In this introduction to programming in Go, we’ll build the service in small steps, making a mess along the way and then cleaning it up again. Along the way, we’ll encounter lots of cool Go features, including:

Struct types and methods

Unit tests and benchmarks

Goroutines and channels

Interfaces and dynamic typing

A Simple Fund

Let’s write some code to track our startup’s funding. The fund starts with a given balance, and money can only be withdrawn (we’ll figure out revenue later).

Go is deliberately not an object-oriented language: There are no classes, objects, or inheritance. Instead, we’ll declare a struct type called Fund, with a simple function to create new fund structs, and two public methods.

fund.go

1

2

3

4

5

6

7

8

9

10

11

12

13

14

15

16

17

18

19

20

21

22

23

24

25

packagefunding

typeFundstruct{

// balance is unexported (private), because it's lowercase

balance int

}

// A regular function returning a pointer to a fund

func NewFund(initialBalance int)*Fund{

// We can return a pointer to a new struct without worrying about

// whether it's on the stack or heap: Go figures that out for us.

return&amp;Fund{

balance:initialBalance,

}

}

// Methods start with a *receiver*, in this case a Fund pointer

func(f *Fund)Balance()int{

returnf.balance

}

func(f *Fund)Withdraw(amount int){

f.balance-=amount

}

Testing with benchmarks

Next we need a way to test Fund. Rather than writing a separate program, we’ll use Go’s testing package, which provides a framework for both unit tests and benchmarks. The simple logic in our Fund isn’t really worth writing unit tests for, but since we’ll be talking a lot about concurrent access to the fund later on, writing a benchmark makes sense.

Benchmarks are like unit tests, but include a loop which runs the same code many times (in our case, fund.Withdraw(1)). This allows the framework to time how long each iteration takes, averaging out transient differences from disk seeks, cache misses, process scheduling, and other unpredictable factors.

The testing framework wants each benchmark to run for at least 1 second (by default). To ensure this, it will call the benchmark multiple times, passing in an increasing “number of iterations” value each time (the b.N field), until the run takes at least a second.

For now, our benchmark will just deposit some money and then withdraw it one dollar at a time.

fund_test.go

1

2

3

4

5

6

7

8

9

10

11

12

13

14

15

16

17

18

packagefunding

import"testing"

func BenchmarkFund(b *testing.B){

// Add as many dollars as we have iterations this run

fund:=NewFund(b.N)

// Burn through them one at a time until they are all gone

fori:=0;i&lt;b.N;i++{

fund.Withdraw(1)

}

iffund.Balance()!=0{

b.Error("Balance wasn't zero:",fund.Balance())

}

}

Now let’s run it:

1

2

3

4

5

6

$go test-bench.funding

testing:warning:no tests torun

PASS

BenchmarkWithdrawals20000000001.69ns/op

ok funding3.576s

That went well. We ran two billion (!) iterations, and the final check on the balance was correct. We can ignore the “no tests to run” warning, which refers to the unit tests we didn’t write (in later Go programming examples in this tutorial, the warning is snipped out).

Concurrent Access

Now let’s make the benchmark concurrent, to model different users making withdrawals at the same time. To do that, we’ll spawn ten goroutines and have each of them withdraw one tenth of the money.

Goroutines are the basic building block for concurrency in the Go language. They are green threads – lightweight threads managed by the Go runtime, not by the operating system. This means you can run thousands (or millions) of them without any significant overhead. Goroutines are spawned with the go keyword, and always start with a function (or method call):

1

2

3

// Returns immediately, without waiting for `DoSomething()` to complete

go DoSomething()

Often, we want to spawn off a short one-time function with just a few lines of code. In this case we can use a closure instead of a function name:

1

2

3

4

go func(){

// ... do stuff ...

}()// Must be a function *call*, so remember the ()

Once all our goroutines are spawned, we need a way to wait for them to finish. We could build one ourselves using channels, but we haven’t encountered those yet, so that would be skipping ahead.

For now, we can just use the WaitGroup type in Go’s standard library, which exists for this very purpose. We’ll create one (called “wg”) and call wg.Add(1) before spawning each worker, to keep track of how many there are. Then the workers will report back using wg.Done(). Meanwhile in the main goroutine, we can just say wg.Wait() to block until every worker has finished.

Inside the worker goroutines in our next example, we’ll use defer to call wg.Done().

defer takes a function (or method) call and runs it immediately before the current function returns, after everything else is done. This is handy for cleanup:

1

2

3

4

5

6

7

func(){

resource.Lock()

defer resource.Unlock()

// Do stuff with resource

}()

This way we can easily match the Unlock with its Lock, for readability. More importantly, a deferred function will run even if there is a panic in the main function (something that we might handle via try-finally in other languages).

Lastly, deferred functions will execute in the reverse order to which they were called, meaning we can do nested cleanup nicely (similar to the C idiom of nested gotos and labels, but much neater):

1

2

3

4

5

6

7

8

9

10

11

12

13

14

func(){

db.Connect()

defer db.Disconnect()

// If Begin panics, only db.Disconnect() will execute

transaction.Begin()

defer transaction.Close()

// From here on, transaction.Close() will run first,

// and then db.Disconnect()

// ...

}()

OK, so with all that said, here’s the new version:

fund_test.go

1

2

3

4

5

6

7

8

9

10

11

12

13

14

15

16

17

18

19

20

21

22

23

24

25

26

27

28

29

30

31

32

33

34

35

36

37

38

39

40

41

42

43

44

45

46

47

48

49

50

packagefunding

import(

"sync"

"testing"

)

constWORKERS=10

func BenchmarkWithdrawals(b *testing.B){

// Skip N = 1

ifb.N&lt;WORKERS{

return

}

// Add as many dollars as we have iterations this run

fund:=NewFund(b.N)

// Casually assume b.N divides cleanly

dollarsPerFounder:=b.N/WORKERS

// WaitGroup structs don't need to be initialized

// (their "zero value" is ready to use).

// So, we just declare one and then use it.

varwg sync.WaitGroup

fori:=0;i&lt;WORKERS;i++{

// Let the waitgroup know we're adding a goroutine

wg.Add(1)

// Spawn off a founder worker, as a closure

go func(){

// Mark this worker done when the function finishes

defer wg.Done()

fori:=0;i&lt;dollarsPerFounder;i++{

fund.Withdraw(1)

}

}()// Remember to call the closure!

}

// Wait for all the workers to finish

wg.Wait()

iffund.Balance()!=0{

b.Error("Balance wasn't zero:",fund.Balance())

}

}

We can predict what will happen here. The workers will all execute Withdraw on top of each other. Inside it, f.balance -= amount will read the balance, subtract one, and then write it back. But sometimes two or more workers will both read the same balance, and do the same subtraction, and we’ll end up with the wrong total. Right?

1

2

3

4

$go test-bench.funding

BenchmarkWithdrawals20000000002.01ns/op

ok funding4.220s

No, it still passes. What happened here?

Remember that goroutines are green threads – they’re managed by the Go runtime, not by the OS. The runtime schedules goroutines across however many OS threads it has available. At the time of writing this Go language tutorial, Go doesn’t try to guess how many OS threads it should use, and if we want more than one, we have to say so. Finally, the current runtime does not preempt goroutines – a goroutine will continue to run until it does something that suggests it’s ready for a break (like interacting with a channel).

All of this means that although our benchmark is now concurrent, it isn’t parallel. Only one of our workers will run at a time, and it will run until it’s done. We can change this by telling Go to use more threads, via the GOMAXPROCS environment variable.

1

2

3

4

5

$GOMAXPROCS=4go test-bench.funding

BenchmarkWithdrawals-4---FAIL:BenchmarkWithdrawals-4

account_test.go:39:Balance wasn'tzero:4238

ok funding0.007s

That’s better. Now we’re obviously losing some of our withdrawals, as we expected.

Make it a server

At this point we have various options. We could add an explicit mutex or read-write lock around the fund. We could use a compare-and-swap with a version number. We could go all out and use a CRDT scheme (perhaps replacing the balance field with lists of transactions for each client, and calculating the balance from those).

But we won’t do any of those things now, because they’re messy or scary or both. Instead, we’ll decide that a fund should be a server. What’s a server? It’s something you talk to. In Go, things talk via channels.

Channels are the basic communication mechanism between goroutines. Values are sent to the channel (with channel <- value), and can be received on the other side (with value = <- channel). Channels are “goroutine safe”, meaning that any number of goroutines can send to and receive from them at the same time.

Buffering

Buffering communication channels can be a performance optimization in certain circumstances, but it should be used with great care (and benchmarking!).

However, there are uses for buffered channels which aren’t directly about communication.

For instance, a common throttling idiom creates a channel with (for example) buffer size 10 and then sends ten tokens into it immediately. Any number of worker goroutines are then spawned, and each receives a token from the channel before starting work, and sends it back afterward. Then, however many workers there are, only ten will ever be working at the same time.

By default, Go channels are unbuffered. This means that sending a value to a channel will block until another goroutine is ready to receive it immediately. Go also supports fixed buffer sizes for channels (using make(chan someType, bufferSize)). However, for normal use, this is usually a bad idea.

Imagine a webserver for our fund, where each request makes a withdrawal. When things are very busy, the FundServer won’t be able to keep up, and requests trying to send to its command channel will start to block and wait. At that point we can enforce a maximum request count in the server, and return a sensible error code (like a 503 Service Unavailable) to clients over that limit. This is the best behavior possible when the server is overloaded.

Adding buffering to our channels would make this behavior less deterministic. We could easily end up with long queues of unprocessed commands based on information the client saw much earlier (and perhaps for requests which had since timed out upstream). The same applies in many other situations, like applying backpressure over TCP when the receiver can’t keep up with the sender.

In any case, for our Go example, we’ll stick with the default unbuffered behavior.

We’ll use a channel to send commands to our FundServer. Every benchmark worker will send commands to the channel, but only the server will receive them.

We could turn our Fund type into a server implementation directly, but that would be messy – we’d be mixing concurrency handling and business logic. Instead, we’ll leave the Fund type exactly as it is, and make FundServer a separate wrapper around it.

Like any server, the wrapper will have a main loop in which it waits for commands, and responds to each in turn. There’s one more detail we need to address here: The type of the commands.

Pointers

We could have made our commands channel take *pointers* to commands (chan *TransactionCommand). Why didn’t we?

Passing pointers between goroutines is risky, because either goroutine might modify it. It’s also often less efficient, because the other goroutine might be running on a different CPU core (meaning more cache invalidation).

Whenever possible, prefer to pass plain values around.

In the next section below, we’ll be sending several different commands, each with its own struct type. We want the server’s Commands channel to accept any of them. In an OOP language we might do this via polymorphism: Have the channel take a superclass, of which the individual command types were subclasses. In Go, we use interfaces instead.

An interface is a set of method signatures. Any type that implements all of those methods can be treated as that interface (without being declared to do so). For our first run, our command structs won’t actually expose any methods, so we’re going to use the empty interface, interface{}. Since it has no requirements, any value (including primitive values like integers) satisfies the empty interface. This isn’t ideal – we only want to accept command structs – but we’ll come back to it later.

For now, let’s get started with the scaffolding for our server:

server.go

1

2

3

4

5

6

7

8

9

10

11

12

13

14

15

16

17

18

19

20

21

22

23

24

25

26

27

28

29

packagefunding

typeFundServerstruct{

Commandschaninterface{}

fund Fund

}

func NewFundServer(initialBalance int)*FundServer{

server:=&amp;FundServer{

// make() creates builtins like channels, maps, and slices

Commands:make(chaninterface{}),

fund:NewFund(initialBalance),

}

// Spawn off the server's main loop immediately

go server.loop()

returnserver

}

func(s *FundServer)loop(){

// The built-in "range" clause can iterate over channels,

// amongst other things

forcommand:=ranges.Commands{

// Handle the command

}

}

Now let’s add a couple of struct types for the commands:

1

2

3

4

5

6

7

8

typeWithdrawCommandstruct{

Amount int

}

typeBalanceCommandstruct{

Response chan int

}

The WithdrawCommand just contains the amount to withdraw. There’s no response. The BalanceCommand does have a response, so it includes a channel to send it on. This ensures that responses will always go to the right place, even if our fund later decides to respond out-of-order.

Now we can write the server’s main loop:

1

2

3

4

5

6

7

8

9

10

11

12

13

14

15

16

17

18

19

20

21

22

func(s *FundServer)loop(){

forcommand:=ranges.Commands{

// command is just an interface{}, but we can check its real type

switchcommand.(type){

caseWithdrawCommand:

// And then use a "type assertion" to convert it

withdrawal:=command.(WithdrawCommand)

s.fund.Withdraw(withdrawal.Amount)

caseBalanceCommand:

getBalance:=command.(BalanceCommand)

balance:=s.fund.Balance()

getBalance.Response&lt;-balance

default:

panic(fmt.Sprintf("Unrecognized command: %v",command))

}

}

}

Hmm. That’s sort of ugly. We’re switching on the command type, using type assertions, and possibly crashing. Let’s forge ahead anyway and update the benchmark to use the server.

1

2

3

4

5

6

7

8

9

10

11

12

13

14

15

16

17

18

19

20

21

22

23

24

25

26

27

28

29

func BenchmarkWithdrawals(b *testing.B){

// ...

server:=NewFundServer(b.N)

// ...

// Spawn off the workers

fori:=0;i&lt;WORKERS;i++{

wg.Add(1)

go func(){

defer wg.Done()

fori:=0;i&lt;dollarsPerFounder;i++{

server.Commands&lt;-WithdrawCommand{Amount:1}

}

}()

}

// ...

balanceResponseChan:=make(chan int)

server.Commands&lt;-BalanceCommand{Response:balanceResponseChan}

balance:=&lt;-balanceResponseChan

ifbalance!=0{

b.Error("Balance wasn't zero:",balance)

}

}

That was sort of ugly too, especially when we checked the balance. Never mind. Let’s try it:

1

2

3

4

$GOMAXPROCS=4go test-bench.funding

BenchmarkWithdrawals-45000000465ns/op

ok funding2.822s

Much better, we’re no longer losing withdrawals. But the code is getting hard to read, and there are more serious problems. If we ever issue a BalanceCommand and then forget to read the response, our fund server will block forever trying to send it. Let’s clean things up a bit.

Make it a service

A server is something you talk to. What’s a service? A service is something you talk to with an API. Instead of having client code work with the command channel directly, we’ll make the channel unexported (private) and wrap the available commands up in functions.

1

2

3

4

5

6

7

8

9

10

11

12

13

14

15

typeFundServerstruct{

commandschaninterface{}// Lowercase name, unexported

// ...

}

func(s *FundServer)Balance()int{

responseChan:=make(chan int)

s.commands&lt;-BalanceCommand{Response:responseChan}

return&lt;-responseChan

}

func(s *FundServer)Withdraw(amount int){

s.commands&lt;-WithdrawCommand{Amount:amount}

}

Now our benchmark can just say server.Withdraw(1) and balance := server.Balance(), and there’s less chance of accidentally sending it invalid commands or forgetting to read responses.

There’s still a lot of extra boilerplate for the commands, but we’ll come back to that later.

Transactions

Eventually, the money always runs out. Let’s agree that we’ll stop withdrawing when our fund is down to its last ten dollars, and spend that money on a communal pizza to celebrate or commiserate around. Our benchmark will reflect this:

1

2

3

4

5

6

7

8

9

10

11

12

13

14

15

16

17

18

19

20

21

22

23

// Spawn off the workers

fori:=0;i&lt;WORKERS;i++{

wg.Add(1)

go func(){

defer wg.Done()

fori:=0;i&lt;dollarsPerFounder;i++{

// Stop when we're down to pizza money

ifserver.Balance()&lt;=10{

break

}

server.Withdraw(1)

}

}()

}

// ...

balance:=server.Balance()

ifbalance!=10{

b.Error("Balance wasn't ten dollars:",balance)

}

This time we really can predict the result.

1

2

3

4

5

$GOMAXPROCS=4go test-bench.funding

BenchmarkWithdrawals-4---FAIL:BenchmarkWithdrawals-4

fund_test.go:43:Balance wasn'tten dollars:6

ok funding0.009s

We’re back where we started – several workers can read the balance at once, and then all update it. To deal with this we could add some logic in the fund itself, like a minimumBalance property, or add another command called WithdrawIfOverXDollars. These are both terrible ideas. Our agreement is amongst ourselves, not a property of the fund. We should keep it in application logic.

What we really need is transactions, in the same sense as database transactions. Since our service executes only one command at a time, this is super easy. We’ll add a Transact command which contains a callback (a closure). The server will execute that callback inside its own goroutine, passing in the raw Fund. The callback can then safely do whatever it likes with the Fund.

Semaphores and errors

In this next example we’re doing two small things wrong.

First, we’re using a Done channel as a semaphore to let calling code know when its transaction has finished. That’s fine, but why is the channel type bool? We’ll only ever send true into it to mean “done” (what would sending false even mean?). What we really want is a single-state value (a value that has no value?). In Go, we can do this using the empty struct type: struct{}. This also has the advantage of using less memory. In the example we’ll stick with bool so as not to look too scary.

Second, our transaction callback isn’t returning anything. As we’ll see in a moment, we can get values out of the callback into calling code using scope tricks. However, transactions in a real system would presumably fail sometimes, so the Go convention would be to have the transaction return an error (and then check whether it was nil in calling code).

We’re not doing that either for now, since we don’t have any errors to generate.

1

2

3

4

5

6

7

8

9

10

11

12

13

14

15

16

17

18

19

20

21

22

23

24

25

26

27

28

29

30

31

32

33

34

35

36

37

38

// Typedef the callback for readability

type Transactor func(fund *Fund)

// Add a new command type with a callback and a semaphore channel

typeTransactionCommandstruct{

Transactor Transactor

Done chan bool

}

// ...

// Wrap it up neatly in an API method, like the other commands

func(s *FundServer)Transact(transactor Transactor){

command:=TransactionCommand{

Transactor:transactor,

Done:make(chan bool),

}

s.commands&lt;-command

&lt;-command.Done

}

// ...

func(s *FundServer)loop(){

forcommand:=ranges.commands{

switchcommand.(type){

// ...

caseTransactionCommand:

transaction:=command.(TransactionCommand)

transaction.Transactor(s.fund)

transaction.Done&lt;-true

// ...

}

}

}

Our transaction callbacks don’t directly return anything, but the Go language makes it easy to get values out of a closure directly, so we’ll do that in the benchmark to set the pizzaTime flag when money runs low:

1

2

3

4

5

6

7

8

9

10

11

12

13

14

15

16

17

pizzaTime:=false

fori:=0;i&lt;dollarsPerFounder;i++{

server.Transact(func(fund *Fund){

iffund.Balance()&lt;=10{

// Set it in the outside scope

pizzaTime=true

return

}

fund.Withdraw(1)

})

ifpizzaTime{

break

}

}

And check that it works:

1

2

3

4

$GOMAXPROCS=4go test-bench.funding

BenchmarkWithdrawals-45000000775ns/op

ok funding4.637s

Nothing But transactions

You may have spotted an opportunity to clean things up some more now. Since we have a generic Transact command, we don’t need WithdrawCommand or BalanceCommand anymore. We’ll rewrite them in terms of transactions:

1

2

3

4

5

6

7

8

9

10

11

12

13

14

func(s *FundServer)Balance()int{

varbalance int

s.Transact(func(f *Fund){

balance=f.Balance()

})

returnbalance

}

func(s *FundServer)Withdraw(amount int){

s.Transact(func(f *Fund){

f.Withdraw(amount)

})

}

Now the only command the server takes is TransactionCommand, so we can remove the whole interface{} mess in its implementation, and have it accept only transaction commands:

1

2

3

4

5

6

7

8

9

10

11

12

13

typeFundServerstruct{

commands chan TransactionCommand

fund *Fund

}

func(s *FundServer)loop(){

fortransaction:=ranges.commands{

// Now we don't need any type-switch mess

transaction.Transactor(s.fund)

transaction.Done&lt;-true

}

}

Much better.

There’s a final step we could take here. Apart from its convenience functions for Balance and Withdraw, the service implementation is no longer tied to Fund. Instead of managing a Fund, it could manage an interface{} and be used to wrap anything. However, each transaction callback would then have to convert the interface{} back to a real value:

1

2

3

4

5

6

7

typeTransactorfunc(interface{})

server.Transact(func(managedValueinterface{}){

fund:=managedValue.(*Fund)

// Do stuff with fund ...

})

This is ugly and error-prone. What we really want is compile-time generics, so we can “template” out a server for a particular type (like *Fund).

Unfortunately, Go doesn’t support generics – yet. It’s expected to arrive eventually, once someone figures out some sensible syntax and semantics for it. In the meantime, careful interface design often removes the need for generics, and when they don’t we can get by with type assertions (which are checked at runtime).

So we’re done, right?

Yes.

Well, okay, no.

For instance:

A panic in a transaction will kill the whole service.

There are no timeouts. A transaction that never returns will block the service forever.

If our Fund grows some new fields and a transaction crashes halfway through updating them, we’ll have inconsistent state.

Transactions are able to leak the managed Fund object, which isn’t good.

There’s no reasonable way to do transactions across multiple funds (like withdrawing from one and depositing in another). We can’t just nest our transactions because it would allow deadlocks.

Running a transaction asynchronously now requires a new goroutine and a lot of messing around. Relatedly, we probably want to be able to read the most recent Fund state from elsewhere while a long-running transaction is in progress.

In the next Go programming tutorial, we’ll look at some ways to address these issues.