Golang pearl: It’s dangerous to go alone!

In Go, the panic function behaves somewhat like an unrecoverable exception: panic propagates up the call stack until it reaches the topmost function in the current goroutine, at which point the program crashes.

This is reasonable behavior in some environments, but programs that are structured as asynchronous handler functions (like daemons and servers) need to continue processing requests even if individual handlers panic. This is what recover is for, and if you inspect the source you’ll see that Go’s built-in HTTP server package recovers from panics for you, meaning that bugs in your handler code will never take down your entire HTTP server.

Unless, of course, your handler code spawns a goroutine that panics. Then your server is screwed. Let’s demonstrate with a trivial example:

1

2

3

4

5

6

7

8

9

10

11

12

13

14

15

packagemain

import(

"io"

"net/http"

)

funchello(whttp.ResponseWriter,r *http.Request){

panic("hi")

}

funcmain(){

http.HandleFunc("/",hello)

http.ListenAndServe(":8000",nil)

}

This server will happily chug along, panicking every time the root resource is hit, but never crashing.

But if hello panics in a goroutine, the entire server goes down:

1

2

3

funchello(whttp.ResponseWriter,r *http.Request){

gopanic("hi")

}

In our web services, we can never really trust a naked use of go. We wrap all our goroutine creation in a utility function we call GoSafely:

1

2

3

4

5

6

7

8

9

10

11

12

13

14

15

16

17

18

19

20

21

import(

"github.com/launchdarkly/foundation/logger"

"runtime"

)

funcGoSafely(fn func()){

gofunc(){

deferfunc(){

iferr:=recover();err!=nil{

stack:=make([]byte,1024*8)

stack=stack[:runtime.Stack(stack,false)]

f:="PANIC: %s\n%s"

logger.Logger.Error().Printf(f,err,stack)

}

}()

fn()

}()

}

Here’s how we use it:

1

2

3

4

5

funchello(whttp.ResponseWriter,r *http.Request){

GoSafely(func(){

panic("hi")

});

}

Not as syntactically sweet as a naked go routine, but it does the trick. The unfortunate thing (which we don’t really have a solution for) is that any third-party code that spawns a go routine could potentially panic– and we have no way of protecting ourselves from that.

John was a development manager at Atlassian, where he led engineering for the Atlassian Marketplace. Prior to that he was an architect at Coverity, where he worked on static and dynamic analysis algorithms. He has a Ph.D. from UC Berkeley in programming languages and type systems, and a BS from Harvey Mudd College. He climbs rocks, ice, small boulders, and the occasional building.