Socket handoff: Concurrent fd sharing for performance and innovation

Excerpt

When different components want to use a shared resource in different ways--such as when they're implemented in different programming languages, or have APIs that aren't trivially compatible--the result is an API design challenge. X desktops today have both Xlib and XCB competing for access to the same network socket, and we needed a design that would let them share. We'll present this design, how we arrived at it, and why it's even more useful than we guessed.

Description

Xlib is over twenty years old, an impressive lifespan for any software project. However, Xlib has not been well-matched to the needs of X clients for some time now. We felt it had earned retirement, so we created a new X-protocol C Binding (XCB) library, providing a simple, small, thread-transparent interface to X. But to support all the applications that haven’t magically ported themselves to XCB yet, we needed a transition plan.

We first tried re-writing Xlib from scratch as a layer on top of XCB. That was hard. So we stopped.

Then we added some hooks to XCB to allow Xlib to remain mostly unchanged but send requests through XCB. That worked, mostly; however, the complexity of this approach led to bugs, ugly hacks in XCB, and far more hacking on Xlib than continued sanity permitted. Adding special code in our clean new XCB library just to keep the old Xlib way working made us sad, and the looming prospect of becoming the Xlib maintainers made us lose sleep. So we stopped that too.

The current versions of Xlib and XCB now use a technique that we call “socket handoff”, cooperatively alternating exclusive access to the X connection instead of dumping Xlib’s requests through XCB. Socket handoff allowed us to delete a pile of buggy, hackish code, while improving performance, and that made us happy. So we didn’t stop.

The socket handoff interface has proven to be useful not just for Xlib, but also for new language bindings to XCB and for experimental X protocol-level hacks. We believe other network protocol implementations would also benefit from this approach.

By the way, you may be running our socket handoff code on your system right now, so why not come hear about the code used by every application on your desktop?

Speaking experience

Speakers

Biography

Jamey Sharp was placed on Ritalin, briefly, in fifth grade. His interests and activities have been varied ever since. His biggest projects have been the Portland State Aerospace Society, a student rocketry club at Portland State University; XCB, a new low-level binding to the X protocol, in the process of replacing Xlib; and Comic Rocket, because his other projects didn’t leave him enough time to read his favorite webcomics without tool support.

Co-conspirators Jamey Sharp and Josh Triplett get up to a lot of miscellaneous hacking mischief together. Much of this hacking occurs while staring at the same screen, and tag-teaming the keyboard. Sometimes this happens with the two of them in different places. We’ll demo our favorite tools and invite audience contributions to the discussion.

Sessions

Co-conspirators Jamey Sharp and Josh Triplett get up to a lot of miscellaneous hacking mischief together. Much of this hacking occurs while staring at the same screen, and tag-teaming the keyboard. Sometimes this happens with the two of them in different places. We’ll demo our favorite tools and invite audience contributions to the discussion.