This article is going to have a different tone from what I've been posting
the past year - it's a proper rant. And I always feel bad writing those,
because, inevitably, it discusses things a lot of people have been working
very hard on.

In spite of that, here we are.

Having invested thousands of hours into the language, and implemented several
critical (to my employer) pieces of infrastructure with it, I wish I hadn't.

If you're already heavily invested in Go, you probably shouldn't read this,
it'll probably just twist the knife. If you work on Go, you definitely
shouldn't read this.

I've been suffering Go's idiosyncracies in relative silence for too long,
there's a few things I really need to get off my chest.

Alright? Alright.

Garden-variety takes on Go

By now, everybody knows Go doesn't have generics, which makes a lot of
problems impossible to model accurately (instead, you have to fall back to
reflection, which is extremely unsafe, and the API is very error-prone),
error handling is wonky (even with your pick of the third-party libraries
that add context or stack traces), package management took a while to arrive,
etc.

But everybody also knows Go's strengths: static linking makes binaries easy
to deploy (although, Go binaries get very
large,
even if you strip DWARF tables - stack trace annotations still remain, and are costly).

Compile times are short (unless you need cgo), there's an interactive runtime
profiler (pprof) at arm's reach, it's relatively cross-platform (there's even
a tiny variant for embedded), it's
easy to syntax-highlight, and there's now an official LSP
server for it.

I've accepted all of these - the good and the bad.

We're here to talk about the ugly.

Simple is a lie

Over and over, every piece of documentation for the Go language markets it
as “simple”.

This is a lie.

Or rather, it's a half-truth that conveniently covers up the fact that, when
you make something simple, you move complexity elsewhere.

Computers, operating systems, networks are a hot mess. They're barely
manageable, even if you know a decent amount about what you're doing. Nine
out of ten software engineers agree: it's a miracle anything works at all.

So all the complexity is swept under the rug. Hidden from view, but not
solved.

Here's a simple example.

Cool bear's hot tip

This example does go on for a while, actually - but don't let the specifics
distract you. While it goes rather in-depth, it illustrates a larger point.

Most of Go's APIs (much like NodeJS's APIs) are designed for Unix-like
operating systems. This is not surprising, as Rob & Ken are from the Plan 9
gang.

On Windows, files don't have modes. It doesn't have stat, lstat, fstat
syscalls - it has a FindFirstFile family of functions (alternatively,
CreateFile to open, then GetFileAttributes, alternatively,
GetFileInformationByHandle), which takes a pointer to a WIN32_FIND_DATA
structure, which contains file
attributes.

We have an uint32 argument, with four billion two hundred ninety-four
million nine hundred sixty-seven thousand two hundred ninety-five possible
values, to encode… one bit of information.

That's a pretty innocent lie. The assumption that files have modes was baked
into the API design from the start, and now, everyone has to live with it.
Just like in Node.JS, and probably tons of other languages.

But it doesn't have to be like that.

A language with a more involved type system, and better designed libraries
could avoid that pitfall.

Out of curiosity, what does Rust do?

Cool bear's hot tip

Oh, here we go again - Rust, Rust, and Rust again.

Why always Rust?

Well, I tried real hard to keep Rust out of all of this. Among other
things, because people are going to dismiss this article as coming from “a
typical rustacean”.

But for all the problems I raise in this article… Rust gets it right.
If I had another good example, I'd use it. But I don't, so, here goes.

This function signatures tells us a lot already. It returns a Result, which
means, not only do we know this can fail, we have to handle it. Either by
panicking on error, with .unwrap() or .expect(), or by matching it against
Result::Ok / Result::Err, or by bubbling it up with the ? operator.

The point is, this function signature makes it impossible for us to access
an invalid/uninitialized/null Metadata. With a Go function, if you ignore the
returned error, you still get the result - most probably a null pointer.

Also, the argument is not a string - it's a path. Or rather, it's something
that can be turned into a path.

Except when they aren't. And paths aren't. So, in Go, all path manipulation
routines operate on string, let's take a look at the path/filepath package.

Package filepath implements utility routines for manipulating filename paths
in a way compatible with the target operating system-defined file paths.

The filepath package uses either forward slashes or backslashes, depending on
the operating system. To process paths such as URLs that always use forward
slashes regardless of the operating system, see the path package.

// Ext returns the file name extension used by path. The extension is the suffix
// beginning at the final dot in the final element of path; it is empty if there
// is no dot.
func Ext(path string) string

It turns out Rust also has a “get a path's extension” function, but it's a lot
more conservative in the promises it makes:

// Extracts the extension of self.file_name, if possible.
//
// The extension is:
//
// * None, if there is no file name;
// * None, if there is no embedded .;
// * None, if the file name begins with . and has no other .s within;
// * Otherwise, the portion of the file name after the final .
pub fn extension(&self) -> Option<&OsStr>

No! It calls components which returns a type that implements DoubleEndedIterator -
an iterator you can navigate from the front or the back. Then it grabs the first item
from the back - if any - and returns that.

The iterator does look for path separators - lazily, in a re-usable way. There is
no code duplication, like in the Go library:

The problem is carefully modelled. We can look at what we're manipulating
just by looking at its type. If it might not exist, it's an Option<T>! If
it's a path with multiple components, it's a &Path (or its owned
counterpart, PathBuf). If it's just part of a path, it's an &OsStr.

Of course there's a learning curve. Of course there's more concepts involved
than just throwing for loops at byte slices and seeing what sticks, like the
Go library does.

But the result is a high-performance, reliable and type-safe library.

It's worth it.

Speaking of Rust, we haven't seen how it handles the whole “mode” thing yet.

So std::fs::Metadata has is_dir() and is_file(), which return booleans.
It also has len(), which returns an u64 (unsigned 64-bit integer).

It has created(), modified(), and accessed(), all of which return an
Option<SystemTime>. Again - the types inform us on what scenarios are
possible. Access timestamps might not exist at all.

The returned time is not an std::time::Instant - it's an
std::time::SystemTime - the documentation tells us the difference:

A measurement of the system clock, useful for talking to external entities
like the file system or other processes.

Distinct from the
Instant type,
this time measurement is not monotonic. This means that you can save a
file to the file system, then save another file to the file system, and the
second file has a SystemTime measurement earlier than the first. In other
words, an operation that happens after another operation in real time may
have an earlier SystemTime!

Consequently, comparing two SystemTime instances to learn about the
duration between them returns a
Result instead of
an infallible
Duration to
indicate that this sort of time drift may happen and needs to be handled.

Although a SystemTime cannot be directly inspected, the
UNIX_EPOCH
constant is provided in this module as an anchor in time to learn information
about a SystemTime. By calculating the duration from this fixed point in
time, a SystemTime can be converted to a human-readable time, or perhaps
some other string representation.

The size of a SystemTime struct may vary depending on the target operating
system.

Well! It exposes only what all supported operating systems have in common.

Can we still get Unix permission? Of course! But only on Unix:

Representation of the various permissions on a file.

This module only currently provides one bit of information,
readonly,
which is exposed on all currently supported platforms. Unix-specific
functionality, such as mode bits, is available through the
PermissionsExt
trait.

$ cargo run --quiet
error[E0433]: failed to resolve: could not find `unix` in `os`
--> src\main.rs:2:14
|
2 | use std::os::unix::fs::PermissionsExt;
| ^^^^ could not find `unix` in `os`
error[E0599]: no method named `mode` found for type `std::fs::Permissions` in the current scope
--> src\main.rs:9:47
|
9 | println!("permissions: {:o}", permissions.mode());
| ^^^^ method not found in `std::fs::Permissions`
error: aborting due to 2 previous errors
Some errors have detailed explanations: E0433, E0599.
For more information about an error, try `rustc --explain E0433`.
error: could not compile `rustfun`.
To learn more, run the command again with --verbose.

How can we make a program that runs on Windows too? The same way
the standard library only exposes PermissionsExt on Unix: with
attributes.

A build constraint is evaluated as the OR of space-separated options. Each
option evaluates as the AND of its comma-separated terms. Each term consists
of letters, digits, underscores, and dots. A term may be negated with a
preceding !. For example, the build constraint:

// +build linux,386 darwin,!cgo

corresponds to the boolean formula:

(linux AND 386) OR (darwin AND (NOT cgo))

A file may have multiple build constraints. The overall constraint is the AND
of the individual constraints. That is, the build constraints:

In practice, it gets old very quickly. You now have related code split across
multiple files, even if only one of the functions is platform-specific.

Build constraints override the magic suffixes, so it's never obvious exactly
which files are compiled in. You also have to duplicate (and keep in sync!)
function signatures all over the place.

It's… a hack. A shortcut. And an annoying one, at that.

So what happens when you make it hard for users to do things the right way?
(The right way being, in this case, to not compile in code that isn't relevant
for a given platform). They take shortcuts, too.

Even in the official Go distribution, a lot of code just switches on the value
of runtime.GOOS at, well, run-time:

But that doesn't work if you're planning on uploading something large, for
example. How many seconds is enough to upload a large file? Is 30 seconds
enough? And how do you know you're spending those seconds uploading, and not
waiting for the server to accept your request?

So, getlantern/idletiming adds a mechanism for timing out if there hasn't
been any data transmitted in a while, which is distinct from a dial timeout,
and doesn't force you to set a timeout on the whole request, so that it
works for arbitrarily large uploads.

The repository looks innocent enough:

Just a couple files! And even some tests. Also - it works. I'm using it
in production. I'm happy with it.

I'm sure all of these are reasonable. Lantern is a “site unblock” product, so
it has to deal with networking a lot, it makes sense that they'd have their
own libraries for a bunch of things, including logging (golog) and some
network extensions (netx). testify is a well-known set of testing
helpers, I use it too!

This one is the meat of the library, so to say, and it requires a few of the
getlantern packages we've seen:

It does end up importing golang.org/x/net/http2/hpack - but that's just because
of net/http. These are built-ins, so let's ignore them for now.

getlantern/hex is self-contained, so, moving on to getlantern/mtime:

That's it? What's why Go ends up fetching the entiregithub.com/aristanetworks/goarista repository, and all its transitive
dependencies?

What does aristanetworks/goariasta/monotime even do?

Mh. Let's look inside issue15006.s

// Copyright (c) 2016 Arista Networks, Inc.
// Use of this source code is governed by the Apache License 2.0
// that can be found in the COPYING file.
// This file is intentionally empty.
// It's a workaround for https://github.com/golang/go/issues/15006

This is known and I think the empty assembly file is the accepted fix.

It's a rarely used feature and having an assembly file also make it
standout.

I don't think we should make this unsafe feature easy to use.

And later (emphasis mine):

I agree with Minux. If you're looking at a Go package to import, you might
want to know if it does any unsafe trickery. Currently you have to grep for
an import of unsafe and look for non-.go files. If we got rid of the
requirement for the empty .s file, then you'd have to grep for //go:linkname
also.

That's… that's certainly a stance.

But which unsafe feature exactly?

Let's look at nanotime.go:

// Copyright (c) 2016 Arista Networks, Inc.
// Use of this source code is governed by the Apache License 2.0
// that can be found in the COPYING file.
// Package monotime provides a fast monotonic clock source.
package monotime
import (
"time"
_ "unsafe" // required to use //go:linkname
)
//go:noescape
//go:linkname nanotime runtime.nanotime
func nanotime() int64
// Now returns the current time in nanoseconds from a monotonic clock.
// The time returned is based on some arbitrary platform-specific point in the
// past. The time returned is guaranteed to increase monotonically at a
// constant rate, unlike time.Now() from the Go standard library, which may
// slow down, speed up, jump forward or backward, due to NTP activity or leap
// seconds.
func Now() uint64 {
return uint64(nanotime())
}
// Since returns the amount of time that has elapsed since t. t should be
// the result of a call to Now() on the same machine.
func Since(t uint64) time.Duration {
return time.Duration(Now() - t)
}

That's it. That's the whole package.

The unsafe feature in question is being able to access unexported (read:
lowercase, sigh) symbols from the Go standard library.

Why is that even needed?

If you remember from earlier, Rust has two types for time: SystemTime,
which corresponds to your… system's… time, which can be adjusted via
NTP. It can go
back, so subtraction can fail.

And it has Instant, which is weakly monotonically increasing - at worse,
it'll give the same value twice, but never less than the previous value.
This is useful to measure elapsed time within a process.

How did Go solve that problem?

At first, it didn't. Monotonic
time measurement is a hard problem, so it was only available internally, in
the standard library, not for regular Go developers (a common theme):

I thought some more about the suggestion above to reuse time.Time with a
special location. The special location still seems wrong, but what if we
reuse time.Time by storing inside it both a wall time and a monotonic time,
fetched one after the other?

Then there are two kinds of time.Times: those with wall and monotonic
stored inside (let's call those “wall+monotonic Times”) and those with only
wall stored inside (let's call those “wall-only Times”).

Suppose further that:

time.Now returns a wall+monotonic Time.

for t.Add(d), if t is a wall+monotonic Time, so is the result;
if t is wall-only, so is the result.

for t.Sub(u), if t and u are both wall+monotonic, the result is computed by
subtracting monotonics; otherwise the result is computed by subtracting wall
times. - t.After(u), t.Before(u), t.Equal(u) compare monotonics if available
(just like t.Sub(u)), otherwise walls.

all the other functions that operate on time.Times use the wall time only.
These include: t.Day, t.Format, t.Month, t.Unix, t.UnixNano, t.Year, and so on.

Doing this returns a kind of hybrid time from time.Now: it works as a wall
time but also works as a monotonic time, and future operations use the right
one.

So, as of Go 1.9 - problem solved!

If you're confused by the proposal, no worries, let's check out the release notes:

Transparent Monotonic Time support

The time package now transparently tracks
monotonic time in each Time value,
making computing durations between two Time values a safe operation in the
presence of wall clock adjustments. See the package
docs and design
document for details.

This changed the behavior of a number of Go packages, but, the core team
knows best:

This is a breaking change, but more importantly, it wasn't before the
introduction of Go modules (declared “stable” as of Go 1.14) that you could
require a certain Go version for a package.

So, if you have a package without a minimum required Go version, you can't be
sure you have the “transparent monotonic time support” of Go 1.9, and it's
better to rely on aristanetworks/goarista/monotime, which pulls 100+ packages,
because Go packages are “simple” and they're just folders in a git repository.

Parting words

This is just one issue. But there are many like it - this one is as good an
example as any.

Over and over, Go is a victim of its own mantra - “simplicity”.

It constantly takes power away from its users, reserving it for itself.

It constantly lies about how complicated real-world systems are, and optimize
for the 90% case, ignoring correctness.

It is a minefield of subtle gotchas that have very real implications -
everything looks simple on the surface, but nothing is.

The Channel Axioms are a
good example. There is nothing explicit about them. They are invented truths,
that were convenient to implement, and who everyone must now work around.

Here's a fun gotcha I haven't mentioned yet:

// IdleTimingConn is a net.Conn that wraps another net.Conn and that times out
// if idle for more than idleTimeout.
type IdleTimingConn struct {
// Keep 64-bit words at the top to make sure 64-bit alignment, see
// https://golang.org/pkg/sync/atomic/#pkg-note-BUG
lastActivityTime uint64
// (cut)
}

The documentation reads:

BUGS

On ARM, x86-32, and 32-bit MIPS, it is the caller's responsibility to arrange
for 64-bit alignment of 64-bit words accessed atomically. The first word in a
variable or in an allocated struct, array, or slice can be relied upon to be
64-bit aligned.

If the condition isn't satisfied, it panics at run-time. Only on 32-bit
platforms. I didn't have to go far to hit this one - I got bit by this bug
multiple times in the last few years.

It's a footnote. Not a compile-time check. There's an in-progress
lint, for very simple cases, because
Go's simplicity made it extremely hard to check for.

This fake “simplicity” runs deep in the Go ecosystem. Rust has the opposite
problem - things look scary at first, but it's for a good reason. The problems
tackled have inherent complexity, and it takes some effort to model them
appropriately.