Has Apache lost its way?

Since its inception, the Apache Software Foundation has had a profound impact in shaping the open source movement and the tech industry at large.

Founded by the developers of the Apache HTTP server and incorporated as a nonprofit in 1999, the ASF has served as an incubator and support structure to dozens of projects that range from the modest to the massive. Subversion, OpenOffice, Tomcat, newcomers Cassandra, Lucene, Hadoop -- all have come of age under the aegis of the ASF and its core principles, informally known as "the Apache Way."

But tensions within the ASF and grumbling throughout the open source community have called into question whether the Apache Way is well suited to sponsoring the development of open source projects in today's software world. Changing attitudes toward open source licensing, conflicts with the GPL, concerns about technical innovation under the Way, fallout from the foundation's handling of specific projects in recent years -- the ASF may soon find itself passed over by the kinds of projects that have helped make it such a central fixture in open source, thanks in some measure to the way the new wave of bootstrapped, decentralized projects on GitHub don't require a foundationlike atmosphere to keep them vibrant or relevant.

The Apache Way: The ASF difference maker

Ask most anyone involved with the ASF, "What sets the foundation apart?" and a common answer will likely dominate: "The Apache Way."

The six tenets of the Way form the core philosophy of the Apache Software Foundation. In the foundation's own words, these are: "collaborative software development; commercial-friendly standard license; consistently high-quality software; respectful, honest, technical-based interaction; faithful implementation of standards; security as a mandatory feature."

Ashish Thusoo, co-creator of Hive at Facebook and currently serving on the Project Management Committee (PMC) for Hive at Apache, expresses the Way as "consensus-oriented communities whose aim is to create high-quality software that leads its field."

Thusoo explains that the ASF's approach is closer to mentorship than actual project management. This consists of "coaching new members on how to develop consensus, the various types of mechanisms to use for voting and project governance, making sure that new members of a project come from a breadth of companies in the industry," Thusoo says.

Arun Murthy, chair of the Apache Hadoop PMC, describes the ASF's "mantra" as "community over code, i.e., people are the lifeblood of the ASF."

The Way also emphasizes the practical over the theoretical. For instance, when incubating a project under the ASF, there is a strong emphasis on working code, not just an idea -- and on donating the resulting code and IP to the foundation "without fearing lock-in for itself or for its users," as Apache itself puts it.

This last detail, the fact that Apache as a whole becomes the custodian of the code, is part and parcel of an approach that also includes the ASF's trademark policy, which is designed to prevent Apache-sponsored projects from having their branding diluted. Being protective of patents may seem counterintuitive for open source projects, but Apache and others have argued for the use of trademarks as protection against predatory behavior.

What's still worth asking, though, is whether a project needs to be assigned to a foundation to be kept both alive and out of the hands of corporate meddlers -- or, for that matter, whether an existing foundation is even needed for such a thing. (Monty Widenius created his own foundation to oversee MariaDB, his fork of MySQL.)

No one-way ticket to success

In practice, the Apache Way is not a one-size-fits-all solution for incubating or supporting open source projects. Much of this is due to the ASF's highly laissez-faire approach to the projects that come under its stewardship.

As Thusoo explains, the ASF may step in "if it feels that the project is blatantly violating the Apache Way," but by and large it provides "infrastructure, the legal guidance, and above all coaching and membership. Seldom does it come to micromanagement."

This approach is a two-edged sword. On the one hand, projects are largely left on their own on a technical level. On the other hand, the Apache Way can come off as "a very regimented, planned form of governance," as Brian Proffitt, adjunct instructor of management at University of Notre Dame, puts it.

"This can be a very good thing, since some projects are in need of organization," Proffitt says. "But it can also cause tension, since the rules and regulations of the ASF may rankle those who see it as a bureaucracy."

Joe Brockmeier, Apache CloudStack PMC member, notes that the ASF is not "magic dust you sprinkle on a project for instant success. If the people doing day-to-day development [on the project] aren't good at building community, or if the project just doesn't appeal to a large enough audience, Apache isn't going to make it magically successful."

Here is where the first real test of Apache's future comes to the fore: Can today's climate of bootstrapped, intensely collaborative open source projects stand to benefit from the Apache Way if the added bureaucracy provides no sure path to greater adoption?

The ASF's open source niche

The answer, of course, is that it depends on the project.

"ASF and open source in general are best suited for wide-ranging platform-type technologies," says Hadoop PMC chair Murthy. "These are the foundational elements of a development and infrastructure community. Some of the most successful Apache projects have been foundations or infrastructure."

Rob Davies, currently of Red Hat and a member of the PMC for Apache Camel, Apache ActiveMQ, and Apache ServiceMix projects, had the latter two projects moved under the wings of the ASF in 2005 "because we wanted to build a larger community, and at the time, the ASF was the only main open source community for middleware."

Davies explains the attraction of putting a project under Apache's care: "[The project members] know that a project will not die if a key developer gets run over by a bus or the company that developer works for gets taken over. [But] in reality this means it's hard to start new projects at Apache and grow a diverse community, as open source projects don't typically work that way."

An open source project typically starts because of the efforts of one or two people, Davies asserts, and attracts contributors only after it shows it has legs. To that end, Davies adds, "the ASF is best suited to established projects that want to benefit from wider exposure and attract a greater diversity," as was the case with Hadoop, which was donated by Yahoo. "If you want to start a completely new project, the ASF might not be the first place to start."

Facebook's Thusoo feels the ASF is best for projects "that are interested in developing a broader community that has representation from a number of companies in the industry. For open source projects that focus really on having the controls with a single entity and that perhaps do not encourage a lot of wider industry participation in terms of authoring, ASF is not really a correct vehicle."

Part of the inflexibility hinted at here may arise from the way Apache's licensing conflicts with the GPL, still among the most broadly used licenses for open source software. The conflicts revolve around "patent termination and indemnification provisions," according to the ASF -- in short, some of the very elements that make the Apache license and community what it is.

While the GPL may in fact be falling out of favor, replaced largely by the Apache License, more permissive licenses like the GPL-compatible MIT License are becoming increasingly popular. GitHub CEO Tom Preston-Werner used his recent OSCON keynote to endorse the MIT License for its brevity and permissiveness -- central virtues for many projects on GitHub's rapidly evolving open source ecosystem.

Competition is your responsibility

Mention of GitHub brings to mind another criticism of the Apache Way: It doesn't automatically spur projects to remain technologically competitive.

Consider Apache HTTP Server, the ASF's original 1995 project, and still one of the foundation's flagship projects. Once responsible for powering the overwhelming majority of websites, it's now experiencing increasing competition from the Nginx server. Created in 2008, Nginx already powers around 15 percent of websites (Apache stands at 53 percent, down from over 60 percent since June 2011), in large part because it uses a different architecture that is said to handle high loads better, is far easier to configure, and is offered under the simpler and more liberal BSD license.

The ASF's version-control system Subversion has also seen challenges on the technical side, having taken a major backseat with developers to Git and especially GitHub, at least in some measure because the distributed nature of Git is a better complement to the work habits of modern developers.

While the ASF helped keep these projects alive and well, the need to keep them technologically competitive falls outside its bailiwick. But many argue this is a feature of the ASF, not a bug.

Apache CloudStack PMC member Brockmeier believes the ASF does want its projects to be "technologically competitive and widely used." But "it's not within the scope of the foundation to take responsibility for that. And I doubt Apache would be as popular as it is, were the ASF leadership to try to dictate to projects exactly how they can do that."

Ben Cherian, Chief Strategy Officer at Midokura, network virtualization company that contributes to Apache CloudStack, agrees. "I don't believe it's the Foundation's responsibility to deal with the changing market and evolution of the projects," he says. "It's the responsibility of each project's community to adapt to the changing winds of technology and market pressure. As with every software project, sometimes there are lifespans associated with these projects, and death and rebirth are part of the normal lifecycle of software. There will be some projects that are wild successes and some that flounder. This isn't a failure of the Apache project. It's just a reality of how software is accepted and adopted by the market."

To that end, any project under Apache's sponsorship must realize on its own how to remain competitive and not let the sponsorship of the ASF suffice for that.

Project politics and its discontents

The ASF has taken heat of late based on how specific projects have been handled. Case in point: OpenOffice.org, which was donated to the ASF by Oracle in June 2011.

Four months after OpenOffice.org changed hands, the ASF published a statement to quell fears about the future of the project and to forestall some criticism already thrown its way. The statement claimed "destructive statements have been published by both members of the greater FOSS community and former contributors to the original OpenOffice.org product, suggesting that the project has failed during the 18 weeks since its acceptance into the Apache Incubator. We understand that stakeholders of a project with a 10+-year history -- be they former product managers or casual users -- may be unfamiliar with the Apache Way and question its methods." Apache went on to cite the Subversion and SpamAssassin projects as "proof that the Apache Way works."

Some criticism might have been due to the way the change in ownership of OpenOffice.org apparently delayed the software's release schedule. A beta of version 3.4 had been offered in April 2011, but the full-blown 3.4 release wouldn't come out until May of the following year. (Version 4.0, with major code contributions from IBM, was released in July 2013.)

Part of the delay was due to relicensing the suite under the Apache license, a time-consuming process. But some of it was more directly attributable to OpenOffice.org, regardless of who sponsors it, having a release-when-ready approach rather than dropping new versions on any fixed schedule. By contrast, the OpenOffice.org sister project, LibreOffice, drops new releases every six months under the LGPL.

Another recent issue involved the retirement of one of the Foundation's smaller software projects, the Apache C++ Standard Library project. Active since 2005, this project had seen its last revision in mid-2008. The chair for the project, Jim Jagielski, stepped down at the end of May 2013; in July, the ASF board voted to retire the project to the Apache Attic, a space "to provide process and solutions to make it clear when an Apache project has reached its end of life."

This move inspired the ire of one of the project's other contributors, Christian Bergström, who had previously volunteered to take over as project chair. He derided the board's choice as "a stupid decision made by bureaucrats" and claimed, "The project still has potential and the lack of vision and belief from the 'board' that those interested could actually achieve it -- it's flat out disappointing." (Bergström declined to comment for this article.)

Jagielski, when asked about Bergström's complaints, replied: "The Apache C++ Standard Library was given numerous opportunities to 'reboot' itself and re-energize the community, but the sad fact is that it was never able to accomplish it." He also noted that Bergström didn't take the right kind of initiative: "As the mailing list and code repo logs show, there was no activity of merit at all [in the project] for a long, long time, and Mr. Bergström certainly had plenty of opportunity to provide some evidence of that potential; even some code commits from him and others would have been a factor."

Brockmeier also points out that retiring a project to the Attic is not meant to be a death sentence. "Retiring projects is one of the reasons I like Apache's approach overall," he says. "Pushing a project into the Attic doesn't in any way hinder people using the code or reviving development at a later date."

Some Attic-bound projects have been moved into new directions. Apache Avalon, for instance, has since become a whole host of subprojects, some maintained by other entities (such as Loom). On the other hand, Apache Harmony, an open source implementation of Java, was sent to the Attic in 2011. (OpenJDK, an entirely separate project along the same lines started by Oracle, has more or less eclipsed Harmony's place.)

ASF and the future of open source

Open source software development is becoming increasingly split between two paths. Down one path lies the world of individually bootstrapped, spontaneously collaborative efforts hosted on GitHub, usually with little formal backing but great enthusiasm and vibrancy. Down the other is the world of commercially sponsored open source, a world the Apache Software Foundation is heavily invested in, as OpenOffice.org, Hadoop, CloudStack, Tomcat, and several other projects show.

The bigger and more complex the project, the more likely it is to need something of the Apache Software Foundation's structure -- and more important, its wealth of corporate contributors -- to keep it hale and hearty. But all this stands apart from keeping a project competitive with its technological neighbors or whether the ASF approach is compatible with the project in the first place.

As Proffitt states, "The ASF is very good at taking big projects that are on their last legs and revitalizing them with organization and resources. But their methodology is less than effective for smaller projects that can and should be more nimble in their processes."

When Noirin Plunkett spoke on behalf of the ASF at OSCON this year, she noted that there was "no negotiation" as far as three requirements of being an Apache-sponsored project goes: using Apache's license, developing by consensus, and having a diverse community ("We don't have projects that have one sole source of contributors").

Statements like these may be behind Proffitt's notions about the "bureaucracy" of the ASF. "[The ASF does] need to put aside some of their strict perceptions of how things are done and listen more to members' issues at times," he says.

Brockmeier highlights further the differences between the ASF and other foundations: "Most foundations don't do much in the way of incubation or dictating any form of governance. The Linux Foundation, for instance, focuses on promoting technologies (Linux mostly, but also has other projects like OpenDaylight and the Xen Project) and a place for several companies to come together. But [the Linux Foundation] doesn't suggest rules for allowing people to have commit access or have requirements for how decisions are made about the projects."

There's little question the ASF has been a boon to the projects suited to it, although keeping those projects competitive remains the responsibility of the project itself. Likewise, while the ASF's rules have been a great source of support to those who need it, it's clear they can be perceived as a stricture rather than a structure. There's no reason for the ASF to try and be all things to all people, and that model so far has served it and its projects well. But it's also clear it's far from the only model in open source town.

Copyright 2018 IDG Communications. ABN 14 001 592 650. All rights reserved. Reproduction in whole or in part in any form or medium without express written permission of IDG Communications is prohibited.