Inside Modern X11 Programming

Over the last few years, XCB has been gradually gaining maturity as a replacement for the venerable Xlib. In this article, David Chisnall looks at the new API and how it improves life for X11 developers.

From the author of

From the author of

A while ago, I wrote about some of the newer features in X11. One that is often overlooked, but still important, s the X C Binding library. To understand the importance of this new library, we need to first understand some of the background of development on X11.

At its core, X11 is a protocol. It defines a series of binary messages that can be passed over a network, through a socket, or via some shared memory, between a client (an application) and a server (a program managing a display). If you want to write a program that will work with an X server, it needs to be able to generate these messages.

Every X server, from the 1980s to the present day, can understand these messages (although earlier ones may only understand a subset). This is from where X gets its interoperability and backwards compatibility. I can run an application on a SPARC64 running Solaris and display it on a PowerPC machine running OS X and Apple's X11, because the communication between the two components is well defined.

Xlib

Most of the time, however, people writing graphical applications don't want to have to think about this. Protocol details are things that even people writing network applications try to abstract away from the rest of the code, and making every GUI developer write their own implementation of the client portion of the protocol would be a disaster.

The canonical way of programming with X, for a very long time, has been Xlib. This library defines a set of interfaces for common tasks, such as creating windows and drawing lines. These map roughly to the protocol but provide a higher level of abstraction.

Unfortunately, over the years it has turned out that Xlib provides exactly the wrong level of abstraction. Very few people want to program with something as low-level as Xlib, preferring more general toolkits such as GTK, Qt, or GNUstep. Even when Xlib was new, people preferred things like Motif.

The only people using Xlib were the ones writing higher-level toolkits. Unfortunately, in many ways Xlib provided too high a level of abstraction for them. In particular, Xlib attempts to provide a synchronous API where possible. This means that an Xlib call will often send a message to the server, and then wait for the return. This makes programming with it easy, but the latency quickly adds up. Often, the code issuing the call doesn't need the answer immediately, even if it needs it quite soon. On a remote connection, this can quickly kill performance.

A small networkfor example, a home LANtypically has a round-trip time of around 1ms. Imagine, for a moment, that you are running a remote application written with Xlib. The round-trip time means that you can, at most, make 1,000 synchronous calls per second, even if both computers are infinitely fast. If you make asynchronous calls, then even doing a small amount of work between sending the original message and blocking waiting for the answer can give a significant speed boost.

One typical example of this is the QueryTree message. This enumerates all the children of a given window. It is typically used by window managers to find all the children of the root window and set up handling for them. With Xlib, the synchronous API means that this call will block for a complete round trip to the server, even though there is no requirement of most window managers for it to complete synchronously; they could just as easily have it run asynchronously via a callback and handle the result when it becomes ready.

Xlib, being designed to support both application and toolkit development, is a byword for bloat. It contains a massive number of convenience functions for every use of the protocol that the designers could think of. Unfortunately, the fact that most applications use a small subset means that a number of these useful-looking functions are now buggy and barely tested.

There are a number of other problems with Xlib, such as the lack of a number of forms of compile-time checking and the difficulty of interfacing it with threaded code.