Friday, March 24, 2006

Interview with Eric Raymond

Today I am publishing an interview with Eric Raymond President Emeritus and Co-Founder of the Open Source Initiative.A hacker himself since the 1970s, Raymond has always taken an interest in hacker culture. When the Free Software Movement took off in the 1990s, therefore, he set out to try and understand how — contrary to all expectations — hackers were able to develop technically superior software, not least the now ubiquitous GNU/Linux operating system. The result was the highly influential essay The Cathedral and the Bazaar, which was published in 1997. When, a year later, Netscape decided to open the source code of its browser (and indicated that it had been partly influenced by Raymond's essay), Raymond and a number of other Free Software supporters responded by founding the Open Source Initiative (OSI) — with the aim of re-branding Free Software as Open Source Software and making it more palatable to the "suits".Today Open Source software has become mainstream, and much of the credit for this must go to Raymond.This is the third of The Basement Interviews, the introduction of which I am publishing on my blog. The full article (including introduction) is available as a downloadable PDF file (see below for details). This is published under a Creative Commons licence.

Open Source

Eric Raymond, President Emeritus and Co-Founder of the Open Source Initiative, speaks to Richard Poynder

Eric Raymond was born in Boston, Massachusetts, in 1957, the oldest of five children. His father worked as a computer programmer for Sperry Univac, which meant that for the first thirteen years of his life Raymond lived in a succession of foreign countries, including Italy, Venezuela and England. On their return the family settled in Pennsylvania.

Raymond was an extremely bright child with a mild case of cerebral palsy, a combination that was to blight his childhood, and made him a victim of frequent bullying. This, coupled with the strict discipline he experienced during his Catholic education, was to make Raymond deeply suspicious of any kind of power, and turn him into an anti-authoritarian. As he puts it: "I had a lot of childhood experiences which demonstrated that the only times people used force was when they wanted to beat me up."

As a young man Raymond’s interests included math, philosophy, and music. Having to constantly adjust to new schools, new languages, and new cultures, he had also developed what he calls his "alien anthropologist mode" — a frame of mind he was later to use to great effect as the self-elected "tribal historian and resident ethnographer" of the Open Source and Free Software movements.

As a freshman at the University of Pennsylvania Raymond was immediately marked out as a potential math prodigy. Having found school insufficiently stretching for his above average talents, however, he lacked the necessary discipline or emotional maturity to cope with the demands of an undergraduate course, and after suffering a "math burnout" left without a degree.

His time at Penn was not wasted however: while ducking classes Raymond taught himself programming, and for the next five years he worked as a programmer for a number of technology companies.

Eventually concluding that he "didn’t fit into the conventional corporate framework", in 1985 Raymond changed direction to become an independent consultant, and freelance writer. His first book project, written while sitting out a year's retainer with his last employer, Rabbit Software, was Portable C and UNIX Systems Programming. Released under the pen name J E Lapin the book was published in 1987.

A close friend of Richard Stallman's in the 1970s, Raymond was an early convert to the Free Software Movement, and contributed to Stallman's GNU Project, although the extent and significance of that involvement is a source of disagreement between Raymond and Stallman today. Raymond also contributed to a number of other Free Software projects, including the Gosmacs editor.

Becoming increasingly interested in hacker culture, in 1990 Raymond took over the Jargon File — a collection of hacker slang originally created at Stanford in 1975.

When the first copies of the GNU/Linux operating system began to circulate in the 1990s, therefore, Raymond was well placed to report on what he quickly saw to be the most significant achievement of the Free Software Movement to date. Indeed, the story he subsequently told about the development of the Linux kernel has become the dominant narrative of the Open Source Movement.

Raymond concluded that what was most radical about GNU/Linux was not so much that it was the first free operating system, but that in developing the Linux kernel Linus Torvalds had invented a totally new method for creating software. Moreover, in doing so he had contravened the cardinal rule of software engineering.

Since the 1970s any programmer worth his salt had implicitly believed Brooks' Law. This held that the only reliable way of developing software was to create small, centrally controlled, hierarchically structured teams of professional developers, and have them beaver away in monkish isolation — a method that Brooks characterised as being akin to the way in which Reims Cathedral had been built .

By contrast, Linux had been created by thousands of geographically distributed hackers collaborating over the Internet. With frequent releases, and constant user feedback, Linux had emerged, said Raymond, out of "a great babbling bazaar of differing agendas and approaches".

Brooks' Law implied that such an anarchic approach would have inevitably ended in chaotic disarray. Instead, contrary to all expectations, the developers of Linux had created a kernel that not only worked, but was technically superior. Raymond concluded that this success was based on the principle that "given enough eyeballs, all bugs are shallow''.

Keen to share his insights, Raymond penned The Cathedral & the Bazaar — an essay that not only offered a compelling explanation of how the Free Software Movement had enabled such a model to arise, but turned out to have articulated something that all hackers knew subliminally, but had never brought to consciousness.

By providing this "generative myth", says Raymond, he gave the community the necessary focus to capitalise on the model that Torvalds and the other Linux developers had accidentally created. The value in doing so, he explains, is that unless you "organise people's perceptions of isolated facts even the most innovative set of innovations may languish in the margins of the economy for a long time."

It turned out that some "suits" also found Raymond's narrative compelling. In January 1998 Netscape released the source code for the Netscape browser, in the hope that a community effort could achieve what Netscape on its own could not: hold back Microsoft's approaching monopoly on the web browser market. As the Netscape press release put it, the aim was "to harness the creative power of thousands of programmers on the Internet by incorporating their best enhancements into future versions of Netscape's software."

To Raymond's immense satisfaction, Netscape executives indicated that their decision had been influenced by The Cathedral & the Bazaar.

Dubbed by Raymond "a shot heard around the world", Netscape's move not only provided an unexpected PR fillip for Free Software, but raised the possibility that it could gain mindshare in corporate boardrooms as well as among hackers.

Believing that a once-in-a-lifetime opportunity had arisen, Raymond concluded that it was essential to "re-brand" Free Software. His fear was that while Fortune 500 companies might be amenable to an ethically neutral argument about efficiency improvements, the Free Software Foundation's moralising about the ethical imperatives of making software free would only alienate them. In February 1998, therefore, Raymond flew to the West coast to meet with a group of like-minded people, and co-founded the Open Source Initiative.

Raymond's message was simple: in a world in which software development is constantly growing in complexity small groups of isolated programmers working in secret can no longer deliver the goods. Today, therefore, success depends on making software code freely available, and allowing as many people as possible to "peer review" and test it. As he succinctly puts it "secrecy is the enemy of quality."

It was a powerful message, and throughout 1998 a growing roll call of influential software companies — including IBM, Sun Microsystems, Oracle, Informix, and Corel — announced initiatives to support Open Source. By the end of the year the Movement had acquired a sufficient head of steam that few could ignore it.

Success, however, has come at a price. Not only has Raymond sacrificed his friendship with FSF founder Richard Stallman, but the Movement has been split. While pragmatic Raymondism focuses primarily on marketing the concept of Open Source, idealistic Stallmanism insists that Free Software is an ethical issue; a matter of right versus wrong. By treating the issue as purely a question of efficiency, says Stallman, Raymondism "is not sufficient to give us freedom that is secure". In short, Stallman believes that since Raymondism lacks the conviction that Free Software is an end in itself, it threatens to subvert the aims of the Movement.

Like Stallman, therefore, Raymond has become a controversial figure. For Free Software groupies he is an egotistical pretender who has sold his community down the river. For the pragmatists he is a liberator who, in the words of Raymond's friend Jay Maynard, has freed the Movement "from the straightjacket that Stallman tried to force on it."

What's the reality? We should not doubt that Stallmanist idealism could never have won over either the boardroom or Wall Street. Nor should we doubt, however, that Raymondism has introduced risks, not least that the Movement could be co-opted by large corporations and exploited for their own ends — a danger all the greater in light of the Movement's internal split.

Perhaps the fundamental question the split raises is the extent to which the anarchic ethos of hackers can co-exist with hierarchical corporate culture. For Stallman the question is hardly relevant, since for him ethical issues are always prior. Raymond, however, believes that corporate culture can be gradually subverted. "In reality", he says, "it is we who are co-opting them. We are changing their culture, their products, and their way of doing things much more than they are changing us."

Herein, perhaps, lies a key issue for the wider free knowledge movement: should its aims be to replace traditional proprietary modes, or to establish peaceful co-existence with them? Can the world change dramatically, or will it simply become more variegated? More importantly, is there a risk that if they try too hard to co-exist with traditional proprietary models, the various free and open movements could be appropriated by them, and emasculated in the process?

##

Interviewing Richard Stallman had felt like being mauled by a bad-tempered grizzly. In preparing to interview Raymond, therefore, I was conscious that, in contrast to the hippy democratic principles espoused by Stallman, Raymond represents Second Amendment libertarianism, and gun rights. Specifically, Raymond is a self-styled market anarchist who believes that citizens have the right to carry guns in order to protect themselves from the government. To a naïve European that seemed a little scary.

I also knew that in 1999 Raymond had famously sent a bruising e-mail threat to OSI co-founder Bruce Perens, who posted the message to a mailing list and indicated that — in the light of Raymond's gun habits — he had alerted the police .

Nor, it seemed, could journalists assume that they would be spared Raymond's hard-man habits. ZDNet columnist John Carroll once characterised engaging with Raymond as akin to taking part in a café debate where Raymond is "the guy at the table trying to take out his opponent's eye with a fork"

Even Raymond's friends were warning me to tread carefully: "Eric cares about some things so deeply that it is hard to hold a rational conversation with him about them," Rob Landley told me.

"If he thinks poorly of you, he'll tell you in great detail and at great length, sometimes whether you want to hear it or not," cautioned Maynard.

What kind of horrors awaited me in interviewing Raymond? I toyed with the idea of flying out to his lair in Pennsylvania, but wimped out, choosing the safer option of a telephone conversation.

In the event, Raymond was a pussycat. He was articulate, reasonable, and very friendly. He was also surprisingly frank, only evading one question. Even when I recklessly suggested that he had proved a Stalin to Stallman's Lenin he simply roared with laughter, replying sardonically: "Comparing me to Stalin! That is the nastiest thing that has happened to me in weeks."

Was it that the chemistry worked in my favour, or had Raymond decided on this occasion to play Mr Nice Guy? When I asked him if his reputation as an irascible interlocutor was justified he replied obliquely: "I invite you to judge by your interaction with me."

But how to judge? Is Eric Raymond the aggressive tub-thumper reaching for a fork to spear his opponent's eye, or the rational and eloquent essayist able to see and articulate the most pragmatic course of action?

Then there is Raymond the hacker; Raymond the gun-toting anarchist; Raymond the SciFi fan; Raymond the "initiate Wiccan priest and coven leader of long standing" ; Raymond the teetotal polyamorist; Raymond the Tae Kwon Do Black Belt; not to mention Raymond the political cynic, for whom democracy "isn’t very interesting".

Once the interview started one thing became clear: it is difficult to have a conversation with Raymond without frequent references to RMS (as Stallman likes to be called online). Indeed, it was hard not to conclude that Raymond has a little bit of a fixation on Stallman — a fixation clearly encouraged by the Open Source community which, says Landley, views Raymond as the "designated alternative to RMS".

Is this fixation mutual? Possibly. When I later e-mailed Raymond some comments Stallman had made about him he copied Stallman into his detailed and self-justificatory reply. This sparked a fascinating e-mail interchange between the two of them, into which I was copied. Judging by these e-mails both men remain very engaged with one another. Sadly, Stallman later decreed that the e-mail conversation had been "off the record".

Interview completed, however, the question remained: who is Eric Raymond? In Revenge of the Hackers he talks of how, when talking to the press, he deliberately sets out to exude "attractive dissonance." To excite journalists, he explains, the trick is to "sound challengingly weird" — to cheerfully discuss guns, anarchism, and witchcraft "while looking as well-groomed, boyishly charming, and all-American wholesome" as possible. He adds that it is important not to fake weirdness but to be genuinely weird.

Nevertheless, it's hard not to feel that much of Raymond's public persona is more spin than substance. At one point in the interview Raymond said that part of his job is to provoke thought, and get people to think about things. He added: "When that is your job description, if you are not pissing off a certain number of people a lot of the time then you are probably not pushing hard enough."

Undoubtedly Raymond has pissed off a lot of people in his time. On the other hand, his well-crafted and persuasive essays have also made a lot of people think.

In the end perhaps it doesn’t really matter whether or not Raymond is weird. In any case, I will leave readers to judge for themselves!

##

If you wish to read the interview in its entirety please click on the link below. I am publishing it under a Creative Commons licence, so you are free to copy and distribute it as you wish, so long as you credit me as the author, do not alter or transform the text, and do not use it for any commercial purpose.

If after reading it you feel it is well done you might like to consider making a small contribution to my PayPal account. I have in mind a figure of $8, but whatever anyone felt inspired to contribute would be fine by me. Payment can be made quite simply by quoting the e-mail account: aotg20@dsl.pipex.com. It is not necessary to have a PayPal account to make a payment.

What I would ask is that if you point anyone else to the article then you consider directing them to this post, rather than directly to the PDF file itself.

I would like to acknowledge the help of the Open Society Institute, which provided a small upfront grant to enable me to get started on The Basement Interviews Project. Further information about The Basement Interviews can be found at the Open and Shut? site.

To read the interview in its entirety (as a PDF file) click here.
A companion interview with Richard Stallman is available here.