Friday, March 11, 2005

Language Design by Consensus

Guido van Rossum recently proposed a small addition to the Python language (any and all predicate functions). In the discussion that followed, he made an interesting comment:

What worries me a bit about doing a PEP for this simple proposal is that it might accidentally have the wrong outcome: a compromise that can carry a majority rather than the "right" solution because nobody could "sell" it.

For those not familiar with Python development, a "PEP" is a Python Enhancement Proposal. Generally, when someone proposes a new feature for the Python core or standard library, they float the proposal on Python-Dev, and if it seems promising, they write a formal PEP, which covers the reason for the feature, the potential effects, a reference implementation, etc. Then after more discussion and informal voting, GvR makes a "pronouncement" accepting, rejecting, or deferring the PEP.

When maintaining any project (but particularly a large infrastructure project like a programming language), there's a natural tension between having a single authoritative architect, and having a democratic process that tries to meet the needs of as many users as possible. If the project leans too far toward the single-architect model, it can wither because it doesn't meet the needs of enough new users; worse, if the original architect loses interest, it can die on the vine. On the other hand, if the project goes fully democratic, it can suffer from loss of focus, bureaucratic slowness, and feature bloat.

As Guido mentioned, ideas can become "sanitized" into a least-common-denominator version based on political will rather than usefulness, aesthetics, or logical correctness. For example, in The Design and Evolution of C++, Bjarne Stroustrup explains why pure virtual methods use the awkward "virtual foo() = 0;" syntax rather than a "pure" keyword: the last standards committee meeting for release 2.0 of the spec was just around the corner, and if he'd suggested adding a "pure" or "abstract" keyword, the resulting debate would have delayed inclusion of abstract classes beyond the impeding release of the spec. In other words, a short-term hack, based on the fear of bureaucratic delays, has been enshrined in the language for all time.

The need to placate and compromise is another source of problems in hyper-democratic projects. At a previous job, we were developing a coding standard by team consensus. On indentation, we were divided roughly equally into a "four spaces, dammit!" camp, a "two spaces, dammit!" camp, and a "I don't care, just make a decision, dammit!" camp. We ended up with three-space indentation, which had the alleged benefit of pleasing no one, but offending everyone equally.

Based on completely non-scientific observations over my career, more projects (both open-source and proprietary) stay coherent and successful when a single architect can make decisions. That doesn't mean an autocratic process: like Linus Torvalds or Dave Cutler, they can have "trusted lieutenants" to whom the single architect can delegate trust and responsibility, but a single vision tends to keep a project's design tighter and less susceptible to unproductive ratholes and non-orthogonality.

Chief architects walk a fine line, of course: they need to exert just enough control to keep the project coherent, but not so much that the project stagnates (or in the case of open source projects, people abandon the project). It's more an art than a science, though. I don't think you can describe hard-and-fast rules for how to do this; on the other hand, there's good money to be made convincing people otherwise.

Disclaimer: I'm not disparaging projects linked to here, and I'm not saying they're inherently broken. Each is successful by some measures, but each has flaws that can be traced to its location in the "autocratic-democratic" continuum.