Why I Program in Go

Go is a fresh new programming language, that has come out of Google and is primarily targeted towards server development. It is developed by some very accomplished computer scientists, like Ken Thompson and Rob Pike. I recently launched a significant new product built with Go at work, and it has proved itself out very well in terms of developer productivity and performance. So much so that many other teams are also giving it a go (oh, how punny this language's name is).

Think about it. How often do you come across a programming innovation that improves productivity and computational performance?

The Path to Go

Before I discuss Go, I would like to talk about how I was led to it. I started working in 2001 and handled multiple languages for the first couple of years while working on short web application development projects. My absolute favourite at that time was C++. In 2004, I started writing lots of code in PHP, non-Object Oriented. Though initially I hated it, especially for the fact that I couldn't catch any errors without running the program first, I learned two important programming lessons for life:

"Object Oriented" is just one way of programming, not the one way of programming. It's not even the best, most of the time.

Scalars (integers, characters, strings, floats, booleans), sequences (lists/arrays) and maps (key-value pairs) as built-ins are enough to support almost all data models.

While PHP was great for writing small scale applications quickly, it was hard to test, hard to debug (too much implicit type conversion) and SLOW. When I thought of taking a break from PHP, I jumped into Java in 2007. Java is perhaps the most depressing language I've ever programmed in. Too restrictive, too inexpressive (in other words verbose) and too "heavy", with all the IDEs, frameworks, build environments etc. that needed to be set up before getting started. I also did not accept the JVM ideology. It's too high a price for phantom portability.

In 2008, I started with Python. I had first played with it about five years earlier but this is when I got on to it full time. I loved Python. It was multi-paradigm, it was very expressive with brilliant stuff like list comprehensions, generators, decorators, **kwargs, etc. Its biggest strength, though, was that all this expressiveness was so cohesively designed into the language that the WriteCode-Run-FixCode-Run... cycle frequently had 0 iterations. New code that I wrote worked as intended more often than in any other language.

There was, however, a problem with Python. It was SLOW. I tried optimisers like Psyco, which transformed into Pypy but didn't deliver too well. Unladen Swallow came and withered.

When I started programming in Python, it appeared that Google had a lot of interest in the language. It's creator, Guido van Rossum, was on their payroll. As days passed and progress on Unladen Swallow slowed, we heard of a new language from Google -- Go. It would have been a bit easier to dismiss Go earlier on as an academic product, had it not been for the reputation of the people behind it, Rob and Ken, and the fact that it came from one of the most hard-core commercial software companies on earth. Back in 2009 when Go was announced, I did not understand much about why it was great, except that it had a lot of modern programming constructs and that it was fast.

Let's hold this thought and go on a tangent for now.

In late 2010, when I moved to Zynga and got back from Python to PHP, I had the time to read up on some stuff. One of the books I read was this book, which offers a kind of gestational walk-through of programming language features using an academic programming language called Oz. Somewhere in its discussion, the book introduces a concept of "dataflow programming" and "declarative concurrency". Now, you might want to follow the preceding link and briefly acquaint yourself with this concept because it is the centre-piece of Go's language features.

When I learned about dataflow programming, I recalled how big a deal it was at Yahoo to be able to fetch data from multiple data sources in parallel and have resolution policies like fastest-source-wins, or wait-until-N-of-M-responses, etc., each with its own timeout handling and so on. The standard solution was a Java daemon with XML documents to describe the intended dataflow. It wasn't pretty. I wished for this capability to exist in a language that, unlike Oz, wasn't academic.

I did not realise at that time that Go was what I was looking for.

Some time early last year we started discussing about trying out other programming languages -- compiled languages, to be more specific -- to get better-than-PHP levels of performance. That's when I again started exploring Go and discovered that it had gathered a lot more backing at Google and gone GA, version 1.0. In the middle of last year, we started bouncing ideas about a new project which required handling concurrency. The PHP shared-nothing architecture wasn't going to work. I thought it was the right time to take Go for a spin.

Working with Go

I got a real feel for programming with Go while working on the project's prototype. Here's what I observed:

Compiler assistance: It felt really good to get back to a compiled language. The compiler not only checks for code that can theoretically result in a runnable program, but also disallows things that can lead to problems later on. This isn't a unique property of Go, but it's there and that's good.

Standardized formatting: A standard tool to enforce formatting rules that is not subject to change based on the team members' or leader's opinions is a welcome feature for lowering the "not my code" mental barrier.

Small language: Go is a very compact language. It's not bursting at the seams with features. It's a lot like C in that regard, minus the "undefined behaviour" proliferation. That makes it easy to learn and start programming with.

Batteries included: The Go standard library has all the bits and pieces needed to write a server program, whether it be a webserver, raw socket server or anything else that requires access to OS built-ins.

Easy concurrency: Go channels and goroutines work very nicely to handle concurrency. The best bit is that unlike Node.js, Twisted, etc., you don't have to write your entire application as event handlers, which makes things easier. Unless you're a hardcore JavaScript programmer.

Easy to jump in: Time to get started with Go is pretty short. Much less than languages that require elaborate frameworks and IDEs to get set up.

Fast time to "done": Productivity with Go is really high. There is so much of common server programming components built into the language or its standard library that you can write fairly complex servers in very little time. Less code also means faster time to correct program behaviour.

Multi-core, High performance: The runtime is surprisingly fast and stable. It feels spectacular, for someone used to writing servers in Python, etc. Unlike Python and Node.js, the Go runtime is capable of spawning multiple OS threads and having them execute in parallel.

Go in Production

Some of the features of Go don't seem to be significant until you deploy it to production. The things that I really love about Go in this regard are:

Single Binary: Go produces statically linked binaries that are self sufficient and would execute on any system that they're built for. This is especially great for cloud based deployments because your deployment node does not need anything beyond a bare-bones OS image. Compare this to the elaborate dependency setup that is required for most other application environments and the benefits become obvious.

Cross Compilation: Earlier I mentioned about the JVM being a bad trade-off for phantom portability. In practice, non-trivial Java programs also require porting for different systems. Coupled with dependency management on target systems, portability of Java programs becomes tougher. The nice thing about Go is that I can have my entire build environment for development and testing on my Ubuntu or OS X system and compile for RHEL in production. All I need to do is copy over the cross-compiled binary and I'm done with portability!

Built-in Profiling: With just a few lines of code, you can have an HTTP end-point that will give you live profiling capability for CPU, memory, goroutines, OS threads, lock contention etc. You can use the go toolchain to hit that URL and profile the application on-demand while it's executing and handling live traffic. That's just brilliant!

Shareable Code: One of the biggest impediments to code sharing is the size of a project, or the amount of code it has. Even if people have access to some codebase, it's not always easy for them to dive in and understand the code. Since it takes far fewer lines of code to do the same thing in Go compared to C, C++ or Java, the code is more accessible to others and therefore more shareable. The limited language spec also means that it's less likely that someone might run into obscure constructs or "advanced features" that some people know of and most don't.

Conclusion

To conclude, I have found Go to be the best server programming language I have tried so far. It combines the elegance of Python with the performance of C and C++. Even if it is only 90% as easy as Python and 90% as fast as C or C++, it still works out to be a winning combination. Its benefits are not just restricted to productivity and performance, though. Go applications are easier to develop and deploy across heterogenous systems and virtualised environments. It's also easier to share and collaborate on Go programs, thanks to a single formatting standard, small language spec and concise code.

Great post! However... I feel your language evolution is a little depressing, because (at least from this post), your only experience with static typing has been in languages with amazingly poor type systems (Java, C++, Go). You should consider branching out and learning some Haskell/Scala/ML where you can get the similar feel of PHP and Python with the advantage of type-safety and type-expressiveness.

http://twitter.com/davidpayne11 david payne

You should try Scala, dude.

eknkc

I liked Scala at first but whenever I look at Scala documentation on some generic method, type signatures make my eyes bleed.

http://blog.maxaller.name/ Max

Well, based on "The limited language spec also means that it's less likely that someone might run into obscure constructs or "advanced features" that some people know of and most don't." I suspect that Scala would make the OP upset as well -- most of the cons of Java, with some syntactic sugar and a language spec hundreds of pages long. I don't particularly like Java, but at least the language is simple.

clementd

Scala 2.9 spec : 191 pages
Java SE 7 spec : 670 pages

roger peppe

scala 2.9 spec: 311K
Go 1 spec: 131K

clementd

For the record, I think this is a completely useless metric. Dismissing scala over its "spec hundreds of pages long" is a bit silly IMO.

In this regard, brainfuck > go

http://tech.t9i.in/ Tahir Hashmi

Size of the language spec is just one sub-part of one of the reasons to choose Go.

Scala is good if you're on the JVM. I'm not sure I'd go for the JVM just to be able use a particular language.

Peter Kariuki

clojure spec 1 page just kidding

roger peppe

There's a clojure spec?

Facebook User

Me too. I also like Scala first which looked simple at first. But after one month I wrote a Scala application, I forgot all Scala syntax except "var/val". It looked horrible for me now.

http://twitter.com/sshillo Sean Shillo

Interesting post. David, it sounds like you prefer Scala over Go, can you give more detail as to why?

http://twitter.com/davidpayne11 david payne

Performance, and there is a framework called LIFT. It's wonderful. The documentation is slightly outdated, but the framework itself is amazing. It's like this framework that lets you 'build and forget'. Contrast this to other frameworks where you worry about scalability once your app hits that level of traffic. It's just plain sweet. Also combine it with an app server, there you go, you can be the CEO/CTO and focus on your business rather than focusing your energy on how to scale up. BUT, Go is a good choice too, just that I haven't seen any mature frameworks in Go that have been tested in production yet. (Correct me if i'm wrong?). LIFT is easily a 5+ year old framework that has stood the test of time.

slvr

"1. Compiler Assistance" - Compiled languages do not necessarily provide the features that you mentioned in this paragraph. You're more in love with statically typed languages, and the type safety it affords with the compile-time type enforcement (which is kind of like a limited form of program verification). See also: http://en.wikipedia.org/wiki/Type_safety

http://tech.t9i.in/ Tahir Hashmi

You said it right.

http://rendion.myopenid.com/ render

Before bashing the JVM with that old canard of non-java programmers that we spend time porting our apps, may I ask if you have written any "non-trivial" go apps? I have yet to hear of one existing.

http://tech.t9i.in/ Tahir Hashmi

I didn't mean to say that Go is more portable than Java. It's more like the JVM doesn't buy much for portability, while adding a layer of virtualization.

http://iadnan.net/ Adnan Hasnain Alam

Using your example, testing on Ubuntu and deploying on RHEL does not mean that you have rid of all the RHEL specific bugs. You are only assuming that RHEL and Ubuntu will behave the same. I'll never trust any stress testing that wasn't done on the target system.

http://tech.t9i.in/ Tahir Hashmi

I didn't say there's no testing on the target system, but the longer develop-test-bugfix cycle doesn't happen on it.

http://iadnan.net/ Adnan Hasnain Alam

That is unfair. Even with JVM, you can do majority of the testing on dev machines. Only some tests - load & stress tests, to be precise - have to be done on the target systems. So this "advantage" isn't real.

http://tech.t9i.in/ Tahir Hashmi

The advantage is that you don't need an extra layer of runtime virtualization to achieve portability.

Go achieves portability by having a higher level of abstraction built into the language and standard library, thereby reducing dependency on OS-specific primitives for stuff like networking, concurrency, etc.

Jeff Schroeder

I have it on good authority from a close friend youtube.com uses (and wrote/maintains) this large nontrivial go app: code.google.com/p/vitess/

http://twitter.com/elimisteve elimisteve

Bit.ly built NSQ in Go and uses it to process 35,000 requests/second. It's designed to handle "billions of messages per day."

The Yahoo java dameon you mentioned wouldn't be DISC/jDISC, would it? I used to run that team, would love to hear more if that is the case.

http://tech.t9i.in/ Tahir Hashmi

No, it was yFed. Couldn't recall the name while writing the blog.

ted stockwell

You are absolutely, totally wrong about Java's portability. I have been writing non-trivial Java apps on Windows and deploying on Unix and Linux for more than a decade with no issues. I dont even think about it anymore.

The last thing I want to do is to go back to compiling for every platform.

thens

Interesting post and for the moment I am pretty pleased with my (small) go applications. One thing confuses me a lot, although it seems that it was intended to be as simple as possible: the whole go packaging and the modules.. I simply don't get it "right".
Do you have any recommendation on how to use packages (or different modules) in productive go code?

http://tech.t9i.in/ Tahir Hashmi

The problem is that while packaging in Go is quite simple, the build errors don't do a good job of guiding you out of a mistake. Seems like small messaging improvements are being made. In the mean time, take a look at the official doc on this matter (http://golang.org/doc/code.html) if you haven't already. The folks at golang-nuts google group are also quite helpful.

Seems like you don't like Go. The decisions against support exception handling in the method you familiar and the RAII pattern is -- intentional. This wasn't a side effect, they did it all very explicitly, I wouldn't expect it to come anytime soon.

Channels are the preferred method for synchronization, and clean up a lot of designs. Deterministic destruction in a garbage collected language makes little sense. Exception handling (in the vein of Java) was obviously consciously avoided.

bap

Single Binary....yeah! Except for system programming, I don't see the need for shared linking with today's hardware.

http://iadnan.net/ Adnan Hasnain Alam

We chose the path of dynamically linked libraries years ago in our product. We've created an unnecessary extra task of dependency management. We've wasted many hours chasing bugs that were traced back to only mismatched libraries.

The big difference is that I found comfort in Java+JVM after going through similar moves. I am yet to build credible experience with Go and so have no opinions on that.

http://twitter.com/JessiDarko Jessica Darko

Mandating that open brackets cannot start on a new line is a decision that lowers code readability, but it is also a very insightful illumination of the nature of the people who created this language: They're petty religious nuts.

I've always preferred brackets get their own line, but never worried about those who thought differently, I could read their code and their choice wasn't harming me.

But in go, my choice has been taken away from me, and that tells me that the creators of go are petty assholes, and god knows what other arbitrary and religious choices they have made in creating the language.

It's as if someone made a programming language and forced everyone to use [vi/emacs] to edit it, such that the code wouldn't even compile if it was edited instead with [emacs/vi]. Whichever you prefer, vi or emacs, surely you can see that it would be asinine to require people to use a specific editor.

roger peppe

The advantage that comes from having a single uniform style is greater, IMHO, than the advantage of being able to use your own style. I've used both braces on their own line and K&R style style, and it's not hard to get used to both.

There's nothing petty about the decision - it leads to real benefits, both for readers of code, and for tools operating on the code (it's trivial to write a tool that processes code through an intermediate AST representation without breaking people's code formatting).

The creators of Go are emphatically *not* "petty assholes" - they are pragmatic programmers that have made some hard choices that lead to a better language and ecosystem overall.

snorble

When you do real work for real money, you realize how petty and immature YOU sound. Having code in a consistent format has real benefits. If you are not experienced enough to see that yet, then please remain silent, rather than removing all doubt. If someone on my team went so far as to call good men assholes over where we place our curly braces, they would not be on the team for long. There is nothing in the world that is healthy to hate.

Steve High

Wow. My path seems almost identical to yours. I moved from PHP to Python then splintered into whatever the job required...C#, Java, and Objective-C.

I love Python, but it suffers from some slowness, albeit not as bad as PHP. I found the .NET and J2EE stacks to be a miasma of over-abstracted namespace hell. Although C# is making very good progress, and each iteration is a definite leap forward.

I love Objective-C for all the reasons most people seem to hate it. I like the syntax, and having separate header files. I like the lack of namespaces and verbosity. I also like using Xcode, so there's that.

When I first downloaded the Go runtime, it was the first public release. At the time I thought it wasn't much more than a novelty, since the functionality was either somewhat limited or just poorly documented. Each iteration is faster and more robust.

Go has pretty much replaced Python as my scripting language for Getting Things Done, just because it is so easy to get up and running.

heapoverflow

+1 for Objective-C

hit9

oh,This shall be shared

http://twitter.com/elimisteve elimisteve

I love Go, too! Over the past 6 months I've transitioned from 80% Python and 20% Go, to something like 75% Go and 15% Python (plus 10% JS). Why? Many of the reasons you've stated.

These days it's almost embarrassing to look at Python code and think, "is this thing seriously only doing one thing at a time? Pathetic!" At-your-fingertips concurrency via goroutines is hard to do without.

#Develop #S40 #Apps and win rewards each & every step of
the way in the #League of Developershttp://www.developerleague.com/?pcode=LOD1&src=10002 and earn
exclusive reward points for everything you do - 10 points for
registration, 1 point each for referring a friend, 1 point each when your
friend registers for this program, 2 points for registering and 5 points
for attending Nokia training camps / Events. And redeem these for exciting
prizes worth over Rs. 10,000.