Go4 and Contributing to Go

with Brad Fitzpatrick

Guests

Brought to you by

Brad Fitzpatrick joined the show to talk about becoming the face of open source Go, getting the community involved in bug triage, the potential future of Go, and other interesting Go projects and news.

Toptal – Hire the best freelance developers and designers with Toptal. Email adam@changelog.com for a personal introduction.

Datadog – Datadog brings you visibility into every part of your infrastructure, plus APM that integrates deeply with Go libraries for monitoring your applications’ performance. With support for Gin, Gorilla Mux, and gRPC, and more on the way, you can get fine-grained performance metrics from your Golang apps with minimal instrumentation. Start your free trial, install the agent, and get a free t-shirt!

I’ve been working on Go for maybe 5-6 years now, and I kind of work on the open source-facing part of the public project, the standard library, I run the build system… I kind of touched a little bit of everything over the years.

So you mentioned one thing that is interesting… There seems to be a set of Go team members who work on internal problems, Go as it relates to projects inside Google that use Go, and then some team members that focus more on the external community level stuff. Is that true?

Yeah, I mean… Well, back in the day there was just the first three or four people, and then maybe like eight or nine people, and everyone kind of did everything, including internal stuff and external stuff. But as Go has grown in popularity both outside and inside Google, we’ve kind of started to specialize more. So there are definitely some people now who never really go into the open source world, and they kind of stay within Google and just work on Google-specific libraries or Google-specific performance problems.

Sometimes you’ll see people make an appearance in the open source world with some miscellaneous bug fix or performance improvement, and that was probably their first foray into public, and they’ve been debugging something for weeks internally, so…

We’re trying to change that, so that Google-only contributors spend more time in the open source community if they kind of feel uncomfortable there… Because there’s different tooling and different processes and stuff… But we’re just kind of getting started on maybe doing that every three months or six months or something, having a week or two of what we’re calling kick-off time, like when the tree opens again for a new Go release, making sure that the Googlers internally are also involved with that.

I’ve noticed that you’ve pretty much become the public face of open source Go. Any issues that come across, you’re the person who’s triaging them; dashboards go down, builders go down - you’re the person that everybody’s talking to. It’s interesting to see how that has shaken out over the years. Is that something that you elected to do, is that something you were elected to do, or did it just kind of happen?

It kind of happened… Then, at some point, Russ or Ian or a group of people just kind of officially decided it was so, and I said, “Okay, well it’s already kind of de facto”, so it was announced officially. That’s kind of how everything in Go happens. You do something long enough and you become the de facto leader; the kind of “you touched it last” problem.

Yeah, that’s a current problem. I’m actually having my first child soon here, and I will be taking some paternity leave. That’s a different type of buzz factor I guess, but I’ve been trying to ramp up some other people to help take over the builders and dashboards and stuff like that.

Yeah, the main room - all the keynotes were livestreamed; the side sessions weren’t livestreamed… But yeah, we’ll still work out details and we’ll probably livestream part of it, so we’ll give you a pass. A wedding doesn’t count, because a wedding you can reschedule… [laughter] But you can’t reschedule the baby.

I know… But you know, I did time it such that it comes out right at the Go 1.9 release, so… [laughter] I don’t know if that’s good or bad, but at least during the 1.9 release freeze I won’t have to worry so much about it.

Actually, it seems Brad does so much of the work of interacting with people… Why don’t we talk a little bit about what the process is like for people to contribute…? There’s a recent blog post by Steve Francia, but I think it’s good to discuss it on the air. By the way, Brad, can you approve my pull request? I just put it in last night… Please? [laughter]

Go is now on its fourth version control system, and I think Perforce was the first one. It was like Perforce, Subversion, then Mercurial and now Git. But throughout all these transitions, we keep updating our tools to feel like the original tool, so all the terminology is still from way back in the day. It’s really weird.

You should probably ping it, because sometimes I lose things, and this is something I need to fix. I need to fix my dashboards, so everyone can help out and see older things. Right now, unfortunately, a lot of the times I only see things that are at the top of my inbox. If you ping it, it will come back to the top.

But in any case, why don’t we hear from Brad what the process is like? You were talking about submitting a proposal… Are there cases where submitting a proposal is not necessary? For example, I submitted that CL, but I had to ask Steve Francia, “Hey, can we add it?” and he said, “Yeah, go ahead and submit a change”, so I did it.

I didn’t open an issue… If I hadn’t talked to Steve, should I have opened an issue first? Or just submitting a CL is fine, if it’s something super small like that?

If it’s super small, just do it. We even made our proposal process be pretty light weight, because we knew that even if we wrote a really formal proposal process, people would ignore it or not read it anyway. So we made the proposal process start out with you just file a bug and start talking, and if we deem that your proposal is too complicated and it requires a little bit more formality, then we send you the proposal template and have you write a formal proposal.

Most of the time, we just immediately say Yes or No, and we don’t make people go through the whole “write a design document” phase.

Okay, thank you. So I actually saw a piece of code that I want to add, but I don’t even have a use for it, personally… I just think, “Wow, it’d just be complete if it existed.” When I submit a proposal, do I need to have a strong use case? Or if I don’t have a use case, can I submit it anyway and say “Hey, I think it should be here”?

We have a fact that I added a while back that was “Why isn’t X in the standard library?” Basically, it says we have too much stuff in this standard library and we kind of made a mistake adding so much, so please do not add more stuff to the standard library unless it’s very important. So generally we would say, “No, please add it on GitHub instead.”

So if you’re proposing adding that thing to the standard library, we would probably be saying No, especially if there’s no reason for it and you just wanna do it for fun. We don’t generally add to the standard library just because we can.

Yeah, like put it somewhere that you can run go get. The problem with the standard library is we can’t really fix things quickly and we can never change things and we can never remove things, so you’re much more flexible if you just add it on GitHub.

Yeah, or there’s other places… You can get your own domain name and make a fancy name for it. I have go4.org for miscellaneous Go utility functions that I need in lots of projects, but I don’t necessarily want to add to the standard library… So I’ll add other cool stuff to go4.org.

Ever since I registered that domain name, go4.org, I get so much spam from people saying, “Hey, I have a similar domain name. Are you interested in purchasing this one?” and they’re all just terrible. They’re all just random letters and numbers and gibberish. I’m like, “Well, it’s true, I did order something that’s pretty much gibberish, but…” No, it’s my inside joke.

I always hate that, when people try to sell you… And the other thing I find too is if you search for a domain and you’re sitting on it, you’re like “I wonder if I should buy that” and you go back, it always seems to become like a premium domain with a higher price, and you’re like “Seriously?!” You have to be prepared to buy right then, otherwise you’re at risk of them jacking the price on you.

[unintelligible 00:12:35.16] seeing how little code I can touch in the tree for 1.9. A lot of stuff I’m doing is really all about getting the community more involved in bug triage and code review triage… Because as Go grows both internally and externally, the number of bug reports we get and the number of changes that people send in just keeps growing and it doesn’t really scale when there’s only a couple of us doing reviews, because we spend all of our time triaging bugs and doing code reviews…

So I’m trying to make it easier for the community to get involved, so they don’t feel intimidated, they know the process, or we have better dashboards telling them what needs attention…

We made this wiki page called Gardening, which is just all the kind of gardening tasks that people can do if you have five minutes or ten minutes to kill and you wanna help Go out. We try to say, “Here’s a list of either GitHub queries for issues that might need attention” or “Here’s the recent code reviews”, “Here’s the sorts of tasks that you can do that might only take a few minutes but it would move something along.”

There’s a list at golang.org/wiki/gardening. Various open source projects use this phrase “gardening” as kind of like a background cleanup task you can do… Because you know, gardening and picking weeds never ends. There’s a whole bunch of tasks listed there.

Maybe we had to label “Waiting for info” on a bug and reviewing the bugs on GitHub labeled “Waiting for info” and seeing “Should this be closed? Did this timeout? Did the OP get back to us with the requested information?” Kind of just moving bugs along and pinging them when necessary…

If someone sends in a code review that has the wrong commit message format, telling them how to format their commit message, telling them that they forgot tests, finding typos… There’s lots of easy things you could do that are unrelated to whether the patch is actually correct, or whatever.

Yeah, so I wonder if some sort of shared presentation like that could be put together and it could be used as content for meetups? A lot of meetups are always looking for content, so each person could present and maybe turn it into like “Hey, we don’t have anything to talk about this month, let’s have a gardening party, where everybody triages some stuff”, and they do it once every other month or something like that, and it becomes a collaborative effort, and people can kind of metor each other, people who have never done it before.

So this like like - correct me if I’m wrong, Brian - taking a couple days and saying “During these couple days…” – well, first of all, the maintainers have to come up with a list of issues to be taken care of. We just do a global effort and say, “During these couple days, we’re going to come together and just smash bugs.”

Yeah, the emphasis though is to create engagement in people who previously may not have had some, so… Rather than spending the five minutes to fix the tiny bug yourself, you spend 20 minutes making it really easy for someone to get in and do it who hasn’t committed patches before.

This is something that’s actually been on my to-do list for the last three months now, I guess. One of the things I was supposed to do last quarter is do a blog posts saying how to contribute to Go and how to get involved with this sort of stuff, but I keep putting it off because I want more dashboards for people to use to make it even easier… But at some point I just need to say enough is enough, “What we have is good enough for people to get involved.”

Yeah, and I think the argument too is I think the Bugmash or – there’s another name for it, Ruby had one… I don’t feel like it was called Bugmash… It was something along those lines. But that’s a lot of times for people who want to contribute the code, and a lot of people when they’re just getting started out aren’t familiar with the codebase and how things work, and they don’t really have the confidence to contribute, so I think you won’t get as many participants in those types of things as you will in like “Hey, you may not wanna jump in and contribute code, but you can contribute in these ways.” By doing so, that will familiarize them with the project and the components and things like that, and then maybe build the confidence that way.

One of the things I’ve seen in meetups too is a lot of people – it takes them a long time to get the confidence to submit a changelist. They’re always afraid that they’re gonna get beat down… And sometimes you do, but people take it different ways. I think helping with the gardening is a way to build that confidence where you’re familiar with the contributors to the project, you’re familiar with who works on what and what are the right questions to ask, and the stages tickets go through, because you’re used to being involved in that process.

I did not know that existed either… But I think any community effort to bring people together and have a little bit of support to get over the hump of getting set up, and having the motivation to get set up, because they know there’s going to be a list of issues that are easy enough for them to do the first time around is huge.

For example, I had never contributed to Go at all, but yesterday I took a little bit of time to get set up, and it was like “I wanna add one line of text, but I have to get through this whole set up”… Which, by the way, was very simple. It took a little bit of time, but it was very simple.

I personally hate our contributing process and that document that’s so long and it scares so many people… There’s not really many steps, but there’s so many words around it that people see it and they just run away screaming and say “Why don’t you use GitHub?”

The answer is we probably should… I mean, when we transitioned to GitHub, it was kind of a really quick, forced migration because code.google.com was shutting down, so GitHub was the obvious place and they had the bug tracker, but we weren’t really ready to give up Garrett, because code reviews on GitHub historically have been pretty terrible. They’ve gotten better over time, but we really liked Gerrit. So we said, “Okay, well we’ll use GitHub for the issue tracker, and the wiki and stuff like that, but the code review - we’ll do it in Gerrit and GitHub will just be a mirror.” But it’s probably time that we wrote tooling to accept GitHub pull requests, and at least for now convert them into Gerrit changes automatically, so the review would happen on Gerrit, Gerrit would still be the upstream… But if someone only knew the GitHub flow, they could stay within the GitHub flow and send new PRs, revisions on the code review and then we just transparently update the Gerrit CL. So probably we need to do that to start, but…

Yeah, because like I was saying, it was not hard to do, but it took a little bit of time, and it was a time that I broke down; like, I would read the paragraph and then do one thing, then go do something else… Like, “Argh! I don’t wanna go through this whole thing!” So I went back and forth, and finally in a few hours I completed all the steps. But it’s something that you have to do one time, and it’s very demotivating if you’re just going through that whole process, the whole text to just do one thing that’s small.

Now that I went through it, I pulled the code, I put down the blog, the website inside the Go project… If I wanna make a documentation change, it’s so simple. I just sync. It’s business as usual. I start a branch, and – I don’t know the terminology, but using the Git terminology, I just push.

Now that I have it there, it’s super simple. I just have to find, “Okay, what contributions can I make, based on what’s out there that you need?” That goes back to my point on having some Bugmash event - that’d be super cool.

People always ask for us to use that more; they want more things… But I’m kind of against that label because we never seem to be able to use it correctly because there’s never really a great definition of what it means. People want like an Easy or a Beginner Friendly label, but the problem is what is easy for one person is totally different for another user.

If something was truly trivial for everyone, it would have been fixed by now. We never really know – and Help Wanted is kind of a silly label too, because we want help on every bug. I’ve been using Help Wanted more and more lately, but I basically just add it to everything… Which is fine, but I don’t know. I feel like people should feel free to jump in and grab anything.

Yeah, there was a project that I contributed to and I can’t remember what it is - it might have been Ruby, or something… Their label was Bite-Sized, or something along those lines, which was kind of more like you could fit it in in one sitting - most people anyway - rather than bigger implementations like Generics - you might do Help Wanted because you want people’s opinion’s on it, but that’s certainly not something a new person is gonna sit down in an hour just to get a contribution…

I wanna say something about the Beginner label… I had an experience - when I was first starting to learn Go, I went to this big open source project, and they had a label for beginners (or Easy or something) and I was going through the issues and I’m like “Holy smokes! This is hard. If this is Beginner… Wow. What is Intermediate? It’s like rocket science kind of stuff.” So it really scared me away, but I’ve been working with programming for a while, so I gave it a chance after I learned, and I can understand this stuff now.

But like Brad was saying, if you label something that sounds like beginner level for you, it could be very complex for somebody else coming in. You can really get people freaked out. You’ve gotta be careful with those labels, but I like the idea of labeling based on bite-size or time chunks… That’s good.

Alright, so we are back, talking to Brad Fitzpatrick. We were talking about Go and contributing before the break, but one thing I wanna lead into is you’ve always got these cool hobby projects going on… What’s something cool you’re working on right now?

I mean, I moved to Seattle recently, so I’ve been working on lots of miscellaneous home automation stuff. We got a bunch of Z-wave light switches, and I got some [unintelligible 00:25:31.23] security cameras and I’ve been working on doing my own motion detection…

Actually, it was the last GopherCon that I gave a lightning talk about my motion detection security system. That’s something like, whenever I have a few minutes I try to improve that a little bit.

No, it’s almost all Go. I have a little Go server in the house that connects to the cameras, and gets their compressed MPEG stream of the video, and then I stream it outside of the house to a cloud instance that has more compute power. Then I have little FFMPEG child_process that decodes the video, and it also does the EDGE detection that is built into the FFMPEG. Then I just output the raw pixels over standard out from FFMPEG and I read the raw pixels out of FFMPEG with the EDGE detection and I compute the delta over time of where the EDGEs move, and once it crosses certain thresholds and certain zones of the video, then it starts recording.

It’s always recording the last 5 or 6 seconds, even when there’s nothing in little rolling ring buffer, but once there’s motion, then I start streaming it to an object on cloud storage.

I have little processes in the background that go and generate GIFs from them, little two-second GIFs, selecting the right frames to make the most interesting GIF possible. Then I can send those on Telegram, or whatever. It’s kind of fun.

That’s awesome. Speaking of projects that are over people’s heads - I don’t know whether I could build something like that that does all the EDGE detection and things like that… That’s awesome. You need random captions for the GIFs, too…

Yeah, I was thinking of throwing it through the Google Vision API or something, and identifying objects… I found some cats that were playing on my front porch the other day, because they set off the motion detection. There was like a black cat and a grey cat that were chasing each other…

When I first moved in, I wanted to do something with that. There were all these cranes in Seattle, and I wanted to have a time-lapse of all these new skyscrapers going up… So I put a camera on the roof, and I have a picture every minute uploaded to a cloud storage. I’m up to like 780,000 of these 8-megapixel JPEGs, so I wanna do something with them. I’ve been trying to think of fun ideas, a little web app to let you drag your mouse around and have the sun follow your mouse, or track planes or boats, or let you sort by hue, so you can get the sky in different colors… But I don’t know, I haven’t spent time with that. I have the data, which was the tedious part, and now that I have it, I need to do something fun with it.

I just sold my car and I found my old iPod… One of those big ones that had a rotating hard drive in it, and it was still in my glove compartment hooked up to the car, because the car had this iPod connector cable in there… And I had basically my college music snapshot on this 80 gig disk, and it hadn’t been updated since then, but it was still working.

I found a box of my earliest programs from the Apple 2 on these five-and-a-quarter-inch floppies, and I had no way to read them, but somebody on the internet said, “Oh, I have the hardware for that. I can image them and send you back the raw data”, then I figured I could put it in the emulator, or something. So I sent him off, and never heard from the guy again, and I tried to ping him every six months and say, “Hey, do you still have the discs? Can you send them back?” Nothing, so…

I still miss some of those days, too… I always ask people when they say they have the five-and-a-halves, it’s like “Do you have [unintelligible 00:31:04.06]?” I betcha you can get that in the emulator now…

I do remember having 28k baud modems. Actually, mine [unintelligible 00:31:42.00] where like if you wanted music, you wanted the whole CD, you had to leave it on overnight and hope that nobody disconnected the phone line… You woke up in the morning, “Noooooo…!”

It’s funny though, because generations now probably don’t even realize the irony. When a lot of us started out with the internet, you called up the internet, right? The internet was served over telephone lines, and now telephone is served over internet lines.

Okay… It follows the late ’70s and ‘80s computer industry, and I think season two is about a dial-up ISP, kind of like a BBS system company that they’re building, and the whole thing is super nostalgic… It’s a pretty good show.

Yeah, so this was the project that got me into Go, actually. I was working on Android at the time at Google, in Mountain View, and the bus ride between San Francisco and Mountain View, on a good day, takes 45 minutes, on a bad day takes like two hours.

So I had sometimes four hours to kill on this bus, and I could either hate my life or I could write code… But I needed something to write, so I had this idea that I wanted to build the storage system to end all storage systems to backup all my stuff and archive all my content from all these social networking sites. I have tweets places and photos places and blog posts in other places… I wanted to have all of my stuff, all my backups and files and content, and all my websites hosted out of this thing.

I knew the data model I wanted, I knew the protocols I wanted; I had kept a bunch of notes… So then I had to actually just build the thing and I had to choose what language I wanted to use, and I googled the options: C++, Python or Java… I had written enough in all of these languages to know I definitely did not wanna write in any of these languages…

C++ is basically only usable if you have a giant standard library like Google and you have a good build system like Google, but otherwise the tooling for C++ is kind of painful. I’d written enough Python and Perl to know that it’s not really great to write servers, because you have to either do lots of callbacks and all this sort of stuff, or you have really terrible performance… And I was writing Java for Android at the time, and I just had enough Java in me that I was just kind of done with it.

This was about the time that Go was coming out, so I decided I would prototype my idea in Go first. I got into that riding on the bus, which was great, because I didn’t need the internet, I had good (enough) tooling at the time… This was before the go commands, so you had to use makefiles, but it still compiled so quickly that it wasn’t too bad… And I kept finding problems of things that were missing in the standard library and things that the HTTP package didn’t get right, so I just started sending changes off to the Go team, which I guess at the time was Ross and Ian and Robert. Ross kept approving my HTTP stuff, and I kept sending more, and eventually I just switched to the Go team full-time.

I got an e-mail from Rob one day, and he was like “Hey, do you wanna do this full-time?” I had already done Android a few years at that point and it was pretty obvious that Android was here to stay… It wasn’t a crazy idea anymore, so it was time for a new crazy idea. But now it looks like Go might be here to stay, too…

So I took a look at Camlistore from watching your videos, and it’s really cool. We have a few questions from listener ZelenHunter; his question about Camlistore specifically is “Will you go back to actively developing Camlistore?” I saw that you did have a release this month, but does that mean you’re actively back to working on it?

No, I actively review stuff that this other guy, Mathieu works on. He works on it all the time. I’m still kind of watching and involved; I don’t write as much code on it as I used to. Maybe when I go on my paternity leave I will do that instead of… In my breaks. We’ll see.

I mean, it’s pretty usable and solid right now. Every release we do gets a little bit more mainstream and a little bit more usable. I think two or three releases ago we did this launcher… If you go to Camlistore.org/launch, we have a little form that creates you your own instance running on Google Cloud.

So then it came out, but it was like at a ugly IP address, and the security was over an HTTPS cert that was self-signed, and then I went off and I worked on LetsEncrypt support and I added that to the Golang. I worked with this other guy who implemented Acme, but then I made this Autocert package that’s a Golang x/crypto/acme/autocert… So we added that to Camlistore as well.

Now Camlistore can get a LetsEncrypt certificate but we needed to do a domain name automatically, so when people created their own instance, they had one. Then we created this DNS server that you get a subdomain of Camlistore.net automatically, and we have a little protocol that’s like Acme, and we’ll give you any subdomain that you can prove that you have the key pair for; so you get like kind of an ugly domain name which is like a fingerprint of your key, .camlistore.net, and we’d automatically do a tie-in DNS server, and then you request a LetsEncrypt cert for that.

The end result is you go through this wizard and you say create vm, and then 40-60 seconds later you have a running instance with a fully-trusted cert and domain name.

Another question from this user is “How is the core Go team dealing with burnout?” I’m not even aware that the Go team has a burnout issue, but I thought it was a good question to ask in case this is the case. Can you talk a little about that?

Yeah, it is getting a little bit repetitive. I know Andrew - he went to work on Upspin because he kind of needed a change… You know, you do the same thing for years and years, and it kind of gets a little repetitive.

I’m kind of there myself, because I think Go 1.9 will be my 10th or 11th Go release, not counting all the point releases in the middle… And it’s a little frustrating that you can’t fix a lot of things and you keep seeing the same bug reports and the same proposals over and over and you have to keep duping all these bugs and say “Sorry, we can’t change this”, or “This would have to wait for some sort of Go 2”…

I try to mix it up and do different tasks occasionally, and focus on different things… The HTTP/2 stuff I did for Go 1.6 was a good distraction, because I got to actually write code and think about new types of problems. Now I’m doing all these dashboards and stats on community interaction, which is interesting in some ways… And I’ve been focusing on Kubernetes defining all of our build infrastructure and all the microservices that run the Go build system… That’s kind of been fun to learn about Kubernetes, but yeah, I definitely go through periods where things are very repetitive and boring, and then I have to find something new to do for a while.

Do you think that the frustration with not being able to change, and things like that, might help motivate more of a 2.0, or at least getting the thought process and talks going on about what 2.0 might look like?

Yeah, I think we’re all kind of feeling that or getting in that – feeling that we probably need to do one. I don’t know, it’s been almost… Go became an open source project in November 2009. I think Rob was saying that if he were to do a Go 2, ten years would probably be a good time.

Personally, my biggest concern is that another language would come out and would have goroutines. I feel like goroutines are Go’s real feature. No one else does lightweight threading really well. Other people have copied the good tooling, but the go type system is incredibly interesting. If someone came out with a language with very good tooling, that was still kind of simple and had goroutines and made writing servers very easy, but it also had a more powerful type system…

Crystal is the Ruby-alike that’s written in C or C++ and it’s fast as hell, compiles down to native… They’ve stolen all of goroutines and channels, the whole works, and it is actually really fast. The standard library still needs some work; it’s not 1.0 yet. I was playing with it a weekend or two weekends ago, and it does feel like Ruby, and it is fast as greased lightning. It’ll be interesting to see…

I mean, you’re right there - goroutines are really awesome. I remember the first time I discovered that you can just use them at will… When you recommend that to people, they’re like “How many of these can I have?” “As many as you need.” “Wait, there’s not some kind of cap?” “No, just keep using goroutines.”

I was following both Rust and Swift… Both of them were flirting with the idea of adding lightweight tasks or goroutines or fibers or whatever you wanna call them, and both projects kind of gave up on it and said, “Well, it’s a little difficult, and I think pthreads are good enough… Maybe we’ll think about this later.”

Everyone kind of keeps thinking about it and not doing it. Somebody else will do it, I’m sure.

Oh yeah, I think it’s a guarantee somebody will do it, it just becomes a question of who - is it a new language, or does it get adopted into the new version of a current language…? But I think it’s hard not to at least consider it with the accelerated growth that Go has seen over just a couple of years.

I think that other prospective new languages at least have to question “Why are people so drawn to that language?” and start to adopt some of those things.

Well, going back to Crystal for a minute… Crystal has the recipe for some good success, because people enjoy the Ruby syntax a lot (some people do) and adding things like goroutines and channels to that seems like it could be a good recipe, but when you play with it, it just doesn’t all click like Go does.

Yeah, I’ve heard many times… One of the core competences of Go is that it’s written to be read, and most languages just aren’t. Go is optimized for people to read that code, and it has such a huge impact on your productivity… It’s hard to describe how hard it would be to replicate that somewhere else.

I think from the other side too, people don’t wanna try the language because they want something they feel is complex… It tickles that part of you brain, like “Oh, I need to learn something super cool and complex”, rather than what they feel is like an easier language. So it’s hard to break that barrier to be like, “No, it doesn’t matter your skill level, you’re still gonna love it. I promise.”

We’re talking about languages now, but I’m dying to ask, and other people are asking, as well… Is there any prospect, and how good of a prospect is there if the answer is yes, for Go to have Generics? I’m asking this because obviously a lot of people want to know, people ask all the time, and my understanding is that there has never been a No to this question. I think the door has always been open to that possibility.

Yeah, I think everybody basically wants it; there’s not very much anti-generic sentiment on the team. I think we would all like it if we could put algorithms in the standard library, and more containers and data structures in the standard library, or somewhere in some shared library, even if it’s not built in the standard library. But there hasn’t been a great proposal…

Ian Lance Taylor has written I think five or six proposals at this point, and every time he generally rejects his own proposals. He has even implemented a few of them… So he’s probably the one that will figure it out. I don’t know if it will be his seventh or eighth proposal he writes… [laughter] But I think he’s getting closer. He seems to like each one of his proposals a little bit more.

Yeah… Whenever there’s some really bizarre UNIX question, like “Here’s a process control group with a TTY session leader and something gets a signal and something dies, blah-blah-blah…”, he’s like “Well, of course… In UNIX, this page of the specification does that, except from that version of UNIX, blah-blah-blah…” He knows how UNIX works and how low-level things work, and he also thinks a lot about languages, so…

The summary is – I imagine if there’s a Go 2, it would have Generics. I don’t think we would do a Go 2 without Generics; it wouldn’t be interesting enough. It would be too big of a change to do a Go 2 and break things without it being worth it. I imagine there probably will be a Go 2 at some point, I just don’t know about timing.

I think any – I mean, I haven’t really thought about the details, but I think if in Go 2 any time you import something, you’re allowed to import down… You’re allowed to import and use a Go 1 package, but Go 1 packages can’t import Go 2 packages, because semantics would be different. I think somehow we have to make the ecosystems coexist, to some degree.

Alright, so we are back, talking to Brad Fitzpatrick. We were just talking about a vision for Go 2. Did anybody wanna jump into any interesting projects and news they may have seen this past week? I’ve got a good one that I saw…

So I hope I don’t butcher his name, but I saw a project called periph.io by Marc-Antoine Ruel. It’s an alternative to Gobot. It doesn’t need any supporting libraries or CGO and things like that… I was chatting with him a little bit about it, and I guess it can do bit banging in the Mhz range. We actually have him scheduled to be on the show in like two weeks, so we’ll get into more detail… But really awesome for doing GPIO and I2C and SPI… You’ve just got one-wire implemented into it too, so if you’re messing around with BeagleBones or Raspberry Pi’s and doing GPIO stuff, that’s a really interesting project to check out.

I know… It stinks, because I decided for the meat probes and stuff, like “Alright, I’m just gonna use just a straight ARM processor, just do embedded C”, and then I saw that, and I’m like, “Aw… I kind of want a single board computer again now.”

So I have a cool project and a shoutout that I wanna make… One of our listeners was listening to GoTime while hiking across the Alps, and sent us an e-mail last week about his code generation tool, because he knows how much I like some code generation…

This code generation tool is at github.com/dave/jennifer, and unfortunately radio is not good for pictures… However, we’ll try to find a way to post the picture of Dave standing on top of some giant Swiss Alps mountain, flashing us the Peace sign because he was listening to GoTime. So a shoutout to you, Dave… That’s awesome. I think GoTime is probably the perfect thing to do to get you through the humdrum of hiking through those boring Alps.

[laughs] Heaven. It sounds like heaven. So that was one. Then the other awesome announcement is from github.com/myitcv/react, and that is React bindings for GopherJS. Now it offers Preact support, which is awesome, because Preact is so much smaller than React… So I am really looking forward to getting some downtime and playing with GopherJS React bindings.

These are specifically cool (these are from Paul Jolly, by the way), because they do code generation for all the yucky stuff. So you just implement a couple interfaces and then type go generate and it generates all of the Go code, which then gets transpiled into GopherJS code, which talks to React or Preact.

Not so much… Mathieu, who works on Camlistore, is starting to transition more of our stuff to GopherJS and to use React, and I kind of look at it and it makes kind of some sense, but I haven’t used much of it myself.

Interestingly, we just talked about that, and what it might need to include, and roughly the timeline… You were kind of saying the ten-year mark probably seems right, but it sounds like discussions would be had about what it needs to look like, and proposals and stuff like that would happen in the meantime.

Oh, interesting. I feel like there’s probably a good amount that can be removed… Probably like 30% or so, based on the things I’ve written and how little I use components. That’s also hard, because how many people use the components that I don’t use…?

Yeah, and that’s where the dilemma comes in, right? Because being in the standard library has its advantage because, for example, I remember starting out doing Go - it was ridiculously easy to do HTTP stuff, because I didn’t have to find a library for it, or write one, or which one of the 15 is the canonical one, that people use? But also, the more that’s in the standard library, the more that has to meet the Go 1 guarantee, right? It’s not able to be changed. So moving it out allows things to move at a quicker pace.

Yeah. The most interesting thing about having things in the standard library is you center the whole ecosystem around certain types. Everyone uses time.time and time.duration, or context.now, you know? Rather than saying, “Oh, which context package do you use?” or “Which time package do you use?”

This is why we even added DatabaseSQL originally to the standard library, because we were watching and there were like four or five databases that had totally different interfaces on GitHub, for MySQL versus SQLite, versus some Postgres… So we added it to Go 1 just to kind of unify the ecosystem to feel the same way. But I don’t know… So what’s the minimum standard library you could have?

I think in answering that question you also have to think about usability, and even this listener, Chris Benson, just mentioned about marketing, how HTTP in the standard library is good marketing for Go. And before even reading that, I was thinking exactly that. One of the biggest hallmarks for Go is “Look, this is such a good server-side language! It even has HTTP right in the standard library. It’s so easy to boot up a server with Go!” So even though it might be a pain to maintain, you have to weigh that out with, okay, it’s how much does it do for attracting people and for usability?

Yeah, I think you you have the usability side of things, but I think that people agree on things… Even, say like the Ruby world - there’s a lot of common packages people use, and those are just the things the community uses, or the agreed upon thing, and they’re not part of the standard library. I think having some of these canonical [unintelligible 01:00:17.00] time and context, and things that are probably gonna be shared. Those types of things I think are needed, not necessarily what you do with them… Like say, for instance, HTTP. Anything that abstracts the network stack, syscalls on the operating system, and Matt Layer in the GoTime FM channel mentioned all of these, too.

Those things I think are needed, especially for adoption, because a lot of people - and you brought up the point too, Brad - aren’t familiar with the Linux internals and how some of these signals are handled, and syscalls, and capabilities and things like that. Abstracting those away definitely helps adoption.

One of the things we have to keep in mind is way back in my earliest Java days, you had to pick the Java framework that you were going to use. Basically, you had to pick your standard library. Was it Apache Commons, was it Java X? And if we take too much out of Go’s standard library, we’re going to fragment the community by forcing groups of those other packages that work well together, and that might be strange.

I think there’s a lot of interesting tools coming out now though, to find packages and tooling, right? A good example, and we talk about it all the time, is – it’s escaping me… The plugin we use in Chrome…

Sourcegraph, yeah. That’s a really interesting thing too, to find packages and see how they’re used and see how many other people are using them. So is the problem really that they have to be in the standard library, or is it that we just need better ways to finding these things?

I think the difficulty comes in maintenance though too, right? If it’s in the standard library - this is kind of playing devil’s advocate back the other way - we can guarantee maintenance. I remember several MySQL drivers in the early days that just got abandoned, and you’re like “Okay, so which one’s popular now? Let’s migrate.”

I think maybe the issue of finding might not be the biggest issue. You might be able to find things, but then you have to decide which one to use, and what criteria do you even use to judge? I don’t know… Just going back to the HTTP case, I’m picking on it - I can’t count how many times I’ve seen people asking “What framework should I use?” and the veterans would say “Just use the standard library, it’s that simple. Even though you have to do a little bit more work to get values or params, just use what’s in the standard library”, and people go “Yeah, okay… Even though there are so many frameworks for HTTP.

So what is the Go team’s stance on the promotion of other libraries outside of the standard library. For instance, as a thought experiment, if these things did not exist, people would probably look to the Go team to point them towards the things they should be using. What’s the Go team’s stance on things like that? If there was another HTTP library that was somehow better, would the Go team direct people towards that? Or do you try to stay out of other people’s projects and the promotion of them?

I mean, we don’t have a policy for or against it. Mostly we stay out of it because we don’t really have the time to do unsolicited code reviews around the community. We used to promote - and we still promote - the miekg DNS library. Whenever someone has some kind of esoteric DNS needs, we say “here is your esoteric DNS package that does everything.” So yeah, we definitely refer people when there’s an answer.

I think as long as there’s guidance from the veterans towards things, that they should be used… I don’t know how much – I kind of wanna go through one night and just look through it and check off ones I don’t think would be useful.

It’d be cool to do a survey through the community and see what everybody agrees on.

It would be interesting to see how much of different packages are just not used at all, or used very little. That’s hard to judge, too… When you think about trying to do the machine learning on that, how do you train it? Maybe some things like HTTP are just inherently used more based on the type of software we write.

I was horrified recently to find that a lot of people are using this package I wrote for – I wrote a memcache client, and it was one of the first things I wrote in Go. It was kind of an exercise in writing a package, and I don’t think I even ever used it myself, but I wrote a bunch of tests and I started a real memcache server as child_process, and the tests, but it doesn’t perform well… It doesn’t pipeline requests on the connection, so if you have lots of activity going on, you’ll get all these new TCP connections to your memcache servers, and stuff.

[unintelligible 01:05:48.13] Google Cloud customer recently, and then some other Google engineers started debugging this library and filing bugs against me that this code that I basically never used was inefficient, and I was like “Crap, now I have to go maintain this thing.”

I mean, they’re in there because I guess Ken really liked them, but if you survey all the code that exists anywhere - inside Google, on GitHub etc. - nobody uses them.

People, for fun, implement the mandelbrot set, or something, but other than that they don’t have very many valid uses.

On the other hand, lots of things relatively use bigints, and those are kind of a pain to use because you can’t use + and - and the built in operators; you have to call all these methods. So we’re kind of in this weird position where complex numbers are first class in Go, but nobody uses them, but big integers used in Crypto and stuff are not easy to use.

There’s another proposal from Rob to make bigints just be automatic. The int type, rather than being 32 bits on 32-bit machines and 64 bits on 64-bit machines, it’s just… An int means a bigint, and you implement it as efficiently and possible, and you have the compiler prove things, so if the compiler can prove it’s never bigger than 64 bit, then it’s actually a 64-bit int, but if it’s unprovable, then it’s a bigint behind the scenes.

I mean, most people don’t use it, so it usually only confuses newcomers, right? Most of us just declare a literal and get the pointer to it. And you find that a lot of people coming into the language, it looks like there’s multiple ways of declaring stuff, right? You can do var name =, you know… Whatever the value is, add & or you can do the quick short declaration operator, or you can do new… I think there’s too many ways, so it’s hard – we always want consistency in the code, right? So that’s probably something I see from newcomers all the time, where it’ll get mixed in this confusion, what they use when, and… Although I do find it kind of ugly when you see an empty struck literal being declared and getting a pointer to it… You’re not initializing it with anything; why are you using the literal? How about you, Brian? Is there anything you’d get rid of, you’d X first thing?

We already have lots of ways to declare memory… I don’t think make adds anything to the bundle, and it just adds confusion. This is coming from someone who teaches Go. Why do we have to make slices and why do we have to make maps, when we declare everything else with var?

Yeah, and it’s just rethinking the way things are declared. There’s just a lot of ways till you run across many of them… A lot of people who have been around the language have their standard way of doing things in different use cases, but…

Yeah, it’s a little unfortunate that naked returns and named result parameters are coupled… But yeah, naked return could probably go. Or let’s say you have a function that returns a time.time, an error, and you just wanna omit – you only wanna return the error, you just want the zero value of everything else; right now you have to say “return time.time{},error”. So there’s various proposals that let you use an underscore or something there to also mean the zero value of any type, or you can also imagine something like return just the error, and omit all the other arguments.

That’s one of the things I’m talking about with the declaration… I don’t really care for the empty struct literal, like the time.time{}… It adds no context, because you’re not initializing with anything. So that’d be really cool, to be able to use just the zero value in kind of a quicker, short form way. I don’t know what that would look like, because I’m not a language designer, but that would be useful.

There’s also some weird things in Go, like ranging over a string gives you the UTF-8 code points, rather than the bytes of the string, and it’s the only thing in the language really that ever assumes that strings are UTF-8. So it’s kind of this weird [unintelligible 01:11:56.22]

I think that’s one of the other big things that needs to be changed, the fact that slices of bytes and strings are so identical, but also so separate that you have to have the whole bytes package and the strings package that are duplicated, and switching between those worlds is so expensive… I don’t know. It’s really gross.

That happens a lot because most packages end up assuming you want to deal with strings so your package takes in your string and then it does a bunch of stuff as a byte slice with it and then returns you a string but the problem is when we start glueing those libraries together we have it wrapped around of this conversion doing from strings and byte slices.

Actually, I had a proposal way back to rechange the language in the whole standard library to assume there was a type that meant a readable view of memory. A string promises that nobody in the world can ever change it including you and a byte slice says you can change it and other people could be changing it but there is no type that basically accepts either a byte slice or a string. So I had a proposal back in the day to add a view of memory type. So you could write a function that accepted either a byte slice or a string and did some operation on it but you were not allowed to write to it and maybe somebody else owned it. So I prototyped the whole standard library with it and Russ even implemented it for a while or he implemented enough of it to decide he didn’t like it. I think the docs are still online at x.godoc.appspot.com. It’s kind of fun to look at what the standard library would look like if the type existed.

Yeah, so this week I wanna give a shoutout to the Changelog website. It’s a piece of art and especially I’d want to give a shoutout to Jerod because he is the main lead on that project and I especially love the Search feature. It makes me so happy. If you are a listener and you are interested in any subject related to Go you can go to the Changelog website - the GoTime website and do a Search for that and there might be one or more episodes that talk about that. It’s fantastic. You may find a reference to it in the shownotes or you may find episodes that talk about that topic.

Well I have done this before but I am gonna do it again because it just makes me happy. Buffalo for web development. It just cannot knock out a website any faster in Go than you can with Buffalo and somebody asked me in the Buffalo channel on Slack today whether anybody had any production use and I thought Gosh I’ve had that GopherCon website in production since the beginning of the year, the GopherAcademy website, the GopherTrain website… I probably got 5 or 6 Buffalo websites in production and they are all just humming along so beautifully.

It seems like the GopherCon and GopherAcademy website are staging environments for any cool project written in Go. We had Caddy running way early and we shouldn’t have had it in production.. and ah let’s launch the GopherCon website with it…

I did not have one this week because I hadn’t been doing a whole lot of new stuff. I just give a shoutout to all the people who are contributing libaries for sensors and things like that for Arduino because on last minute notice Brian and I were able to throw together something very quickly to stream barbecue data for when we do our pig.

Alright so with that a big thank you to everybody on the show especially to Brad for coming on. Huge shoutout to our sponsors for today Toptal and Datadog. Definitely share the show with fellow Go programmers, coworkers, friends. You can subscribe by going to GoTime.fm we are on Twitter. If you have suggestions for topics, questions for guests or you wanna be on the show hit us up on github.com/GoTimeFM/ping.. And with that, goodbye everybody and we’ll see you next week!