20 Years of X

On June 19, 1984, MIT’s Robert Scheifler sent an email to a group of computer scientists working on the university’s Athena project. “I’ve spent the last couple of weeks writing a window system for the VS100,” he wrote. “I stole a fair amount of code from W, surrounded it with an asynchronous rather than a synchronous interface, and called it X… Anyone who wants the code can come by with a tape. Anyone interested in hacking deficiencies, feel free to get in touch.” Little did Scheifler know that this seemingly simple message would later go down in history as the first public release of one of the most popular and versatile windowing systems ever built.

Much has changed about the X Window System (or just “X”) since that fateful, first day. The underlying protocol has undergone eleven major revisions, implementations of the architecture have been modified to run on dozens of operating systems beyond Unix, and its user base has gone from one — just Scheifler — to millions and perhaps even tens of millions, as developers have whole-heartedly adopted the system as the platform of choice for GUI-based applications.

Surely, without X, both Linux and Unix never would have taken the lead in enterprise computing. But with the system’s 20th anniversary just around the corner, more and more critics are beginning to ask the same question: Can X meet the demands of GUI-intensive applications for another 20 years?

From A to X

There is little doubt that the success of X so far is due to its networked architecture, which allows applications to run on one machine while the display is served by another. Although there are proprietary utilities that can emulate this feature on other popular windowing systems, no other windowing system makes networked displays so easy to use.

Most Linux and Unix users already know that redirecting the graphical output of an X-enabled program is a simple matter of setting the DISPLAY variable on any networked machine to the IP address and port number where the desired X server can be reached. Interestingly, this architecture is somewhat backwards when compared to traditional client-server systems in which the server process generally runs a the physical server. With X, the server process actually runs the workstation where the display is being served, while the client runs on the physical server or other machine, as shown in Figure One.

Figure one: The X architecture often seems backwards to those who are used to having server processes run on the remote physical server. With X, the server process runs on the workstation where the graphics are to be displayed.

But not everyone thinks that the X architecture is a wonderful thing. Critics often argue that the network capabilities in X should no longer be included by default in popular X implementations like the open source XFree86 Project. Having all the network code, they say, only hinders system performance for those who are simply using X to display graphics locally.

Whether these complaints are entirely objective is questionable, though. In truth, XFree86 and other implementations don’t actually use network sockets for transferring data locally as long as the DISPLAY environment variable is set correctly. When DISPLAY is set to “.0,” indicating that the display data should be piped to the local display hardware, the system uses kernel sockets rather than TCP/IP. (Of course, the client software must also support data transfer through kernel sockets.) The performance in this case is dramatically better than when DISPLAY is set to 127.0.0.1:.0 or some other network address that would add network latency and bandwidth issues to the system. In fact, recent benchmarks by Xi Graphics (even when taken with a grain of salt due to the fact that Xi Graphics makes its money from Accelerated-X) have shown that graphics displayed locally on Accelerated-X and XFree86 4.2 are just as fast as, or faster than, graphics displayed on Microsoft Windows 2000.

That’s not to say that X is perfect. Certainly, X is still missing support for a handful of features that Apple and Microsoft have had for at least a couple of years now. For instance, systems from both of those companies already allow applications to run within transparent windows; meanwhile, X developers are still discussing the best way to implement such a feature. Similarly, anti-aliased fonts were only recently added to X’s repertoire, and even then there has been much debate over whether it was implemented in the most efficient manner.

But for the most part, these are small technical issues. The X architecture is flexible enough to allow extensions as new features become available, and has managed to keep up quite well. (See sidebar entitled, “State of the Union.”) Indeed, the real trouble with X seems not to be with the technology that goes into the project, but with the management that oversees it.

Growing Pains

When Scheifler created the MIT X Consortium in 1988 to direct the growth of X, he envisioned a member-funded organization that would oversee the development and maintenance of the standard. And for the most part, that vision has remained intact, even now that the responsibility for the standard is in the hands of the Open Group’s X.org. What’s changed, though, is that the incredible popularity of the XFree86 implementation has essentially resulted in the creation of a second governing body: the XFree86 Project.

X.org Chairman Steve Swales admits that this is the case, but believes that there is a defining line between the two projects. “The role of X.org is to bring stability to the standard and to provide a channel for individuals and members to have a say in the specification,” says Swales. “The role of XFree86 is to be a cutting edge innovations body. It’s a symbiotic relationship.”

Not everyone agrees that the relationship is so clear cut. A sizeable portion of the discussion on XFree86′s recently unveiled open forum (forum@XFree86.org) has been focused on defining the specific responsibilities of XFree86 in comparison to X.org. Likewise, the relationship between the two organizations hasn’t always been friendly. In 1998, just two years after it assumed responsibility for the X protocol and brand, the Open Group attempted to release the sample implementation of X11R6.4 under a restrictive licensing scheme. Realizing that this would have placed restrictions on XFree86 development, the XFree86 team backed out all of their associated code. Many Linux vendors supported the XFree86 decision and the Open Group decided to abandon the licensing scheme six months later.

“We were a little disappointed that they didn’t tell us about it in advance,” recalled XFree86 President David Dawes in a 1999 interview with the University of Sydney’s Matthew Arnison. “We basically said, ‘Well it’s not acceptable to us.’ And they said, ‘Oh that’s probably the best we can do…’ I have no doubts that we made the right decision. The only thing I regret is the sort of time we spent. We could have been doing other things.”

The tables were reversed when the XFree86 team integrated Unicode support into one of their previous releases. According to Swales, this was done without “adequate public support” and contradicted existing APIs for UTF-8 support on Solaris and Unix. “We still haven’t sorted that out. It’s not part of the X standard and it’s not very appropriate,” says Swales. Dawes, on the other hand, says he still hasn’t seen the specific details of the problem. “The type of Unicode support introduced at that time wasn’t supposed to cause any incompatibilities,” he says. “If it did unintentionally, then that’s something that should be resolved.”

Meanwhile, the administration of XFree86 has been afflicted with its own internal problems. For one, its six-member Board of Directors has long been seen as a closed and conservative oligarchy by many in the open source community. And complaints to the separate Core Team about bugs, slow releases, or lost contributions have allegedly fallen on deaf ears. For the occasional message that does get through, the response has often been apologetic but firm, stating to the effect that there are too few developers and too much work for the team to get to everything. (Surprisingly, for a while last year, a FAQ on the project’s Web site turned away developers who could have helped alleviate some of the pressures, noting that the group was “no longer accepting applications at this time.”)

Frustrated with the amount of time it took for new video drivers to be added to releases of XFree86, Red Hat’s Mike Harris raised the level of public discussion about XFree86′s management woes on January 9th of this year, when he posted his concerns on Advogato.org, a site popular with free software advocates. In addition to making suggestions for implementing an open bug tracking system, Harris called on developers to organize their own, public XFree86 projects that would allow the community to get involved in building specific X components without having to involve the XFree86 core team until the very end of the process.

The number of responses agreeing with Harris’s post was surprising even to him. “I figured a handful of people would read my last entry and offer a few comments… however I received over 100 emails, as well as numerous [messages] in IRC from various people,” wrote Harris just a few days after his first post. “It is clear to me now that there are a significant number of people out there who are interested in XFree86 development or in contributing to the project in one way or another, [and] that we do have the possibility of building up a community around XFree86, if we really want to.” The XFree86 team responded to the feedback by reconfiguring its mailing lists so that both the private development list, devel@xfree86.org, and the private bug report list, xfree86@xfree86.org, became open to the public. They also decommissioned the advice list, xpert@xfree86.org, making xfree86@xfree86.org into a general discussion forum.

And the push for change didn’t stop there. Concerns about XFree86′s barriers to entry and overall lack of direction ran all the way up to the Core Team, where some members privately discussed the possibility of forking the project and managing their own version of XFree86. The discussions were led by Keith Packard, a longtime X developer who was once said to have touched more X code than anyone else.

“XFree86 is the keystone to the Linux desktop,” Packard told us in an interview conducted soon after he made his feelings public. “However, XFree86 hasn’t managed to keep pace with other projects in growing a developer community or incorporating changes needed by projects dependent on it. I believe the fundamental reason is that the XFree86 community is too insular and it’s development processes are too monolithic.”

When the Board heard rumors that Packard had been considering a fork, they considered it a threat and ejected him from the Core Team. “What Keith has done is among the most low-class, unprofessional, and tactless things I have ever experienced in my professional career,” wrote David Wexelblat, a board member and one of the original XFree86 founders, on the open forum@xfree86.org list. “While still a member of the XFree86 Core Team, he has explicitly attempted to subvert XFree86 by soliciting individuals and corporations to create an alternative to XFree86, including inviting certain Core Team members to join him.”

Packard argues that he withheld his concerns only because he was still culling advice on how best to approach the Board. “We were working out details on how to engage the XFree86 board in making the fundamental changes we [thought] necessary when that body heard misleading rumors about my intentions and decided to publish them,” says Packard.

The Board’s reaction helped solidify Packard’s impression that the management of XFree86 needs to be in the hands of the development community rather than those of a group that is mainly inactive in day-to-day development. “Aside from [board president David Dawes], the current board of directors has no involvement in the development of the X window system,” says Packard. “I thought it irresponsible for a fundamental part of our environment to be lead by people without a commitment to open source software and the X window system.”

The controversy that erupted after Packard’s actions — and the Board’s response to them — became public and is chronicled on various community message boards. Many developers expressed their support for Packard, who they see as a “developer’s developer” — someone who has consistently managed to produce high-quality code year after year, without falling prey to the ranks of upper management. Others offered suggestions for creating a better XFree86, and still others debated the pros and cons of a fork. Few seemed to support the Board outright.

To make matters worse for the Board, its members responded to public messages with comments that were often inconsistent and defensive. For instance, despite excoriating Packard for his actions, Wexelblat noted that he believes “X is obsolescent” and called for a liberal overhaul of the way XFree86 works. Dawes, on the other hand, seemed to encourage Packard to start a fork simply to end all the controversy. “I fail to understand why, if you don’t like the way this particular group of people does things, you don’t go off and do it differently yourself. There’s nothing stopping you,” he wrote. Later, though, he took a different tone. “I’ve got to admit that I think all these issues, comments, clarifications [sic] coming out is great,” he posted, along with the announcement that he was setting up a mechanism for tagging regular snapshots of the CVS trunk — a concession to the “openness” that Packard called for.

Eventually, Packard and his supporters — a group that includes Red Hat’s Harris, X co-founder Jim Gettys, and Gnome founder Miguel de Icaza — decided to launch XWin.org, a site that extends the features offered at XFree86.org and encourages community involvement in each step of the development process. Although Slashdot and other open source outlets have referred to this move as a sign that an XFree86 fork is underway, Packard insists that he has no such intentions. “The X development community needs to work together to form a government,” he warns, or else X is headed for a “success disaster.”

A Bleak Future?

While the events of the past six months have raised a number of important questions within the X community, they have done little in the way of providing answers. On top of it all, the community now has three organizations vying for control over X’s future. Can all three work together? Swales seems to think so.

“I’m very happy to see the members of the X community getting excited about a unified governance of X,” he said. “X.org is actively working with XFree86 and the other interested parties to be a productive part of that new governance.”

For Dawes, though, this whole exercise has served as a reminder of XFree86′s original goals. “XFree86 is taking this opportunity to refocus on what it has traditionally considered important: high performance, working, and portable code, and being an independent organization,” he said, adding that Packard’s “fast track appointment” to the Core Team three years earlier was an example of how the organization had strayed from its original goals.

As for XWin.org, the upstart has already managed to build a strong following. And although the organization has no formal control over either the X specification or XFree86, the influence of dozens of active developers and community leaders can certainly be a powerful thing. Additionally, Packard has already begun to voice goals for X whereas others have not been so articulate.

“Over the last year or two, Jim Gettys and I have come to believe that the Linux desktop is poised for a dramatic increase in the number and scope of uses,” he said. “The suite of applications has matured to the point that Linux can now successfully replace closed-source offerings in a wide variety of environments.” No doubt, going head-to-head with closed-source rivals is a plan that many open source advocates will rally for if called upon.

There’s no sign as to when the three organizations will come to any sort of consensus on what to do next. But one thing’s for certain: they must act soon or risk losing the momentum that has propelled the project for so long. In essence, the fate of X depends on the ability of leaders within the three organizations to create a roadmap that the community agrees with. If they fail, they will be leaving the door wide open to the competition.

State of the Union

In addition to enabling graphics to be displayed across a network, X supports extensions that allow it to take on new features without breaking the underlying, standard protocol. This has contributed greatly to the system’s success over the years, especially in the face of competition from windowing systems like Windows XP and Windows 2000 that are rich in eye candy. Here’s an update on some of the latest and most important extensions to X.

RENDER

Developed by XWin.org creator Keith Packard before he was kicked off the XFree86 Core Team, the Render extension was added to the standard XFree86 distribution in 2001. The extension overcomes a number of limitations in the existing rendering architecture, which was mainly written for early text-based displays, and has generally been welcomed by those looking to make the most of modern GUI features such as anti-aliasing. There are still pockets of critics, though, who argue that Render is incomplete because it puts the burden of font management on the client and not on the server where they believe it should be.

Packard admits that he originally intended to have font management reside in the X server, but that he became convinced later that it was an overly complicated idea. “Client-side fonts turn out to be a huge win,” he says. “This overwhelming task of specifying the bounds of font and text support for future X applications was lifted from my shoulders and the minimal support in the Render extension needed to manage client-side fonts was written down in about an hour or so.”

In addition to making the development of the extension easier, Packard says the decision has resulted in side benefits, like the ability for clients to print using the same fonts that are on-screen.

XINERAMA

The Xinerama extension allows users to create a large virtual display by using two or more physical monitors at a time. This is similar to the multiple display feature that Microsoft added to its operating systems beginning with Windows 98.

When Xinerama was first introduced in 2001, there were conflicting implementations, and so an X.org taskforce was created to help put the specification through a standards process. The public review period for the proposed standard ended last year, and according X.org Chairman Steve Swales, the taskforce is close to wrapping up.

XPRINT

The Xprint extension allows applications to send output to a printer, fax, or graphic document, like PDF or Postscript. The architecture essentially calls for another X server — one that’s connected to printers and so on — running alongside the traditional display-based X server. Applications connect to both servers as needed.

The downside to the Xprint extension is that it relies on the core X rendering model which hasn’t been scaled to meet the demands of modern graphics. Packard and developer Carl Worth are working on a new rendering infrastructure (which includes the Render extension) that will provide high-quality graphics to the screen and printer through a common API.