Proposal: New channels for Rust's standard library

Mar 2, 2019

Two exciting performance improvements are coming to Rust’s standard library soon -
we’re replacing mutexes with parking_lot
and replacing hash maps with hashbrown.
The public interface will stay the same while the internal implementations are swapped
out for much faster ones.
All Rust programs using those primitives will therefore magically get faster, too!

In this blog post, I’m proposing we also replace the guts of
mpsc with
crossbeam-channel for some more performance wins.
However, unlike with mutexes and hash maps, this change will also enable oft-requested new
features that make it tempting to deprecate mpsc altogether and introduce better
channels designed from scratch.

The mpsc channels are not perfect. We have
a bunch of regrets
over some decisions made before their stabilization. Now could be a good time to
revisit those decisions and consider fixing the mistakes.

The dilemma

Just a quick reminder worth bringing up first. Unstable feature
mpsc_select
was introduced in 2015 and we’ve decided to deprecate it in late 2018 with the
intention of removing in a future Rust release.
The reason is that it adds a lot of complexity that feels like
shouldn’t belong to the standard library. The feature never worked
as well as we hoped and crossbeam-channel is a better alternative anyway.
I fully support this decision
as selection makes channels at least 2x more complex when measured in lines of
code, and users who need selection can always reach for crossbeam-channel.

We probably don’t want to copy the whole crossbeam-channel into std
because it is just way too big. Without tests, comments, and blank lines,
it contains ~4100 lines of code, while mpsc contains ~2300, and even that is plenty.
We’d instead prefer a slimmer version of crossbeam-channel that doesn’t support
selection. I have a tentative new implementation for mpsc
based on crossbeam-channel that is only ~1700 lines long. You can find it
here, but keep in mind it’s still a work
in progress.

I’m sure everyone wants mpsc to be faster because why not. However, there might be
disagreements on how far to take this proposal - besides improving performance,
we could also improve the interface and enable new features:

One option is uncontroversial: swap the codebase behind mpsc and
finally implement Sync for Sender.

The other option is more radical: introduce std::sync::channel module
with the new implementation and a modern API. std::sync::mpsc then becomes obsolete so
we’d probably want to deprecate it at some point.

While the second option might seem a bit too far-reaching, I’ll try to make a
convincing case for it and demonstrate the benefits would be worth it.

The summary of the argument is the following. The whole mpsc module is unnecessarily
complicated to use, uses odd jargon, is generally a poor example of design in Rust,
and if we were to do it from scratch today, we’d do it much differently. I personally
even feel it’d be better to have no channels than keep mpsc as is - it is that bad!

Why have channels in std at all?

It has sometimes been suggested we deprecate mpsc and point users to external channel
crates like crossbeam-channel. While this is a compelling option, I think in 2019
channels are a fundamental synchronization primitive, and we do need them in the
standard library. They are basically as important as mutexes! Every modern programming
language should have at least very basic channels in its concurrency kit.

And if we’re going to have channels in the standard library, what should be the
difference between those and channels in external crates? My position is the
standard library’s channels should focus on:

Simplicity. The interface must be lean and simple, and the codebase must be
understandable. We don’t want crazy optimizations that make the codebase
too challenging to maintain.

Good performance. It doesn’t have to be best-in-class but has to be reasonable.
Just wrapping a VecDeque inside a Mutex to make the queue concurrent
would be disappointing.

Fast compilation. The current implementation monomorphizes so much code you
can notice mpsc increasing compilation times!

The “no-brainer” proposal

If we’re to keep mpsc as the channel module in std, then we should at
least incorporate performance improvements from crossbeam-channel and
implement Sync for Sender.

More concretely, this is the conservative proposal I don’t expect anyone to
have objections to:

Delete the unstable and deprecated mpsc_select feature.

Swap out the guts of mpsc with the slimmed-down version of crossbeam-channel.

Add unsafe impl<T: Send> Sync for Sender<T> {}.

Do a crater run to make sure there are no regressions.

If we go this route, the benefits will be:

Faster channels overall. In particular, bounded channels become much faster.

Shorter compilation time for code using channels - a hello world with channels goes
from 1.6 sec down to 1.2 sec on my machine.

There will be some drawbacks, too, but they’re relatively minor:

Bounded channels use more memory - every slot in the buffer contains an
additional AtomicUsize.

Unbounded channels are slower to construct and send a single message.
Creating a channel, sending a message, receiving a message,
and dropping the channel goes from 83 ns to 265 ns.
The current mpsc implementation has a special optimization for “oneshot” channels
that can’t be used anymore. Unfortunately, this optimization also prevents Sender
from implementing Sync.

We can have multiple consumers now!

The new implementation will incidentally also make it possible to implement Sync
and Clone for Receiver trivially. Currently, if one wants to share the receiving side
of a channel, they have to jump through hoops.

You can see this in the
Graceful Shutdown and Cleanup
chapter of The Book: the receiving side is of type Arc<Mutex<mpsc::Receiver<Job>>> and
we receive messages with receiver.lock().unwrap().recv().unwrap(). This is not pretty
code at all but makes sense since the “SC” part in “MPSC” stands for single-consumer.

But why is mpsc a single-consumer channel anyway? Why didn’t we go with
multi-consumer channels from the beginning? I haven’t been involved with Rust
at the time so cannot be completely sure about the real reason, but I believe it boils down
to fast unbounded multi-consumer channels being difficult to implement without
garbage collection.

Unbounded channels have to be represented as a linked list, so receive operations
need to load the head node of the list and do a compare-and-swap to change the head
pointer to the next node. But reading the next pointer in the head node is dangerous
if there’s a concurrent receive operation that might pop this node and deallocate it!

If we were to write a multi-consumer channel in Java or Go, we’d say “whatever” and let
the GC handle deallocation. But Rust offers no such luxury. The usual solution
for us is to use epoch-based garbage collection, but it’s a too big and complex piece
of code to have in the standard library.

Fortunately, it’s possible to implement a multi-consumer channel with a relatively simple
trick
that allocates nodes in segments and occasionally locks
segments for a very short time. This keeps good scalability of channels, simplifies
the code, and even makes it faster in the typical case due to smaller overhead incurred
by GC-related book-keeping. But this trick wasn’t well-known when mpsc was created.

The API needs improvement

Let’s take a good look at the current API for mpsc channels. I’ll omit iterators
and error types because we designed them right the first time and they’re uninteresting
for the sake of this analysis.

Note how SyncSender is strictly more powerful than Sender - it has a
superset of features. And really, there is no good reason why we need two distinct
sender types and a single receiver type. If we were to design channels from scratch
today, I’m sure there would be just a single Sender type.

As already mentioned, another issue with this API is the fact that Sender
doesn’t implement Sync, which will be fixed by the new channel
implementation.

The third issue is the lack of send_timeout method on SyncSender, which would
block for a limited time when the channel is full. If Receiver has recv_timeout,
why wouldn’t there be a similar method on the sending side? I believe the omission
of this method is just an oversight and we don’t have it only because nobody has
implemented it yet.

Confusing terminology

Here’s a line of code from The Book, chapter
Message Passing,
under heading Creating Multiple Producers by Cloning the Transmitter:

lettx1=mpsc::Sender::clone(&tx);

Okay, so we’re creating a new transmitter by cloning the sender
side of a multi-producer… wait, that’s three different words describing the
same concept in a single line!

Synonyms like that make it for more painful user experience than it has to be,
and we’re just getting started. Let’s take a complete tour through all the issues
of the terminology used by mpsc.

Synonyms, shorthands, acronyms

Is it sender, transmitter, or producer? Is it receiver or consumer?
To make matters worse, in Servo, senders are called chans and
receivers are called ports. That’s a lot of words to remember.

The shorthands for sender and receiver are tx and rx. Why not
just use s and r instead? When I was learning Rust, I had to
google for “tx”, read an article on Wikipedia to
learn it stands for transmission in telecommunications, and then connect the dots
to realize it’s a synonym for sender.

And what about the cryptic name mpsc? I can’t count how many times I saw it
misspelled as mspc or something of that sort. It’s not an acronym that
rolls off the tongue nor is it easy to remember. We could’ve named the module
chan or channel instead.

Synchronous vs asynchronous

There are three types of channels:

Unbounded channels - they don’t have a fixed capacity.

Bounded channels - they have a fixed capacity.

Zero-capacity channels - the capacity is zero (a special case of bounded channels).

Now let’s see what these three types are called in various libraries.

Unbounded

Bounded

Zero-capacity

Golang

N/A

asynchronous/buffered

synchronous/unbuffered

std::sync::mpsc

asynchronous

synchronous

rendezvous

futures-channel

unbounded

bounded

N/A

crossbeam-channel

unbounded

bounded

zero-capacity

I think the most frustrating part here is that what mpsc calls synchronous,
Golang calls asynchronous. Oof. The logic is probably in that mpsc thinks of
bounded channels as
synchronous when full and Golang thinks of them as asynchronous when not full.
Both of them are correct in their own ways, but this is a real mess.

We’re about to get async/await soon, so I expect we’ll start talking about asynchronous
channels in a completely different context, which will make the confusion even worse.

And we’re not done yet! Note that when we, e.g. say SyncSender implements Sync, trait
Sync has absolutely nothing to do with the prefix Sync. The Sync trait means
it can be shared by reference across threads, while the Sync prefix means it’s
a bounded channel.

We should’ve called channels bounded and unbounded instead. That way, there’d
be no chance of mistaking those words for anything else.

Disconnect vs close

When all senders or all receivers associated with a channel get dropped, the channel
becomes disconnected, meaning no more messages can be sent into it. I think use
of the word disconnected is perfectly fine here, but it’s unfortunate how pretty much
everywhere else channels get closed instead. While this may seem like a minor
annoyance at worst, it’s becoming more and more of a problem.

First of all, within the codebase of mpsc, channels get closed. Only in the
public documentation they get disconnected.

In crossbeam-channel, I use disconnected just because mpsc uses the same,
but I’m seriously considering switching to closed before publishing version 1.0.

In futures-channel,
channels get closed, but senders and receivers can also be manually disconnected,
which essentially means handles become “null” and using them will result in a panic.
Therefore, disconnect in
mpsc and in futures-channel does not mean the same thing at all, which is really confusing!

Finally, disconnected is a bit longer than closed - consider
RecvTimeoutError::Disconnected. That’s wordy and not fun to type.

The “clean slate” proposal

If we’re going to introduce new channels rather than try fixing mpsc, let’s
revamp the interface entirely and avoid all the mistakes previously made.

I suggest we take the following steps in that case:

Create std::sync::channel module with new channels.

Change the guts of mpsc to use channel behind the scenes, but otherwise don’t change it.

Do a crater run to make sure there are no regressions.

Stabilize std::sync::channel.

Phase out mpsc and nudge users towards channel.

Here’s how the new channels would look. There is only a single sender type
for both bounded and unbounded channels:

Note how beautifully symmetrical Sender and Receiver now are. And this API is more
powerful than mpsc despite being smaller and simpler!

The new channels would use clearer terminology:

There’s no mention of producers, consumers, or transmitters.

Also no mention of synchronous and asynchronous channels - they are bounded and
unbounded instead.

Senders and receivers are abbreviated as s and r.

Channels get closed rather than disconnected.

If you’d like to see the full interface with examples, check out the
documentation
generated from the prototype and compare it to the
documentation
for mpsc channels.

Conclusion

Despite all the flaws, the mpsc module is a brilliant piece of code
and was one of the coolest and most advanced channel implementations at the time
Rust 1.0 came out. But state of the art has progressed since then and I believe
it’s time for it to go.

Our terminology around mpsc channels is all over the place. It’s a hindrance to
learning, and I think the unnecessary clunkiness paints a bad picture of Rust.
Now is a great chance to come up with a new
and well-thought-out language for talking about channels that will be adopted throughout
the library ecosystem. Currently, every channel library has its own set of
annoying inconsistencies deviating from others, making the whole situation even worse.

I’m aware adding new APIs and removing old ones from the standard library is going to
be painful. But my opinion is keeping the status quo is the worse scenario and
being stuck with poorly designed channels will be even more painful down the road.
I hope we fix the mistakes in our channels and the sooner that happens, the better.

If we decide to transition to the new channel module, I promise to help by:

Writing clear instructions in the docs on how to switch from mpsc to channel.

Refreshing The Book with new idioms. Do we accept pull requests?

Updating the Rust Cookbook.

Pushing all channel libraries to follow suit by using consistent naming with the
new channels.

And, of course, if we decide to stick with mpsc, I’ll still swap the codebase for the new one
and add the missing features like Sync implementation for Sender and the
send_timeout method.