If you are someone who is accustomed to working in a traditional, proprietary software development, the open source development model might challenge many of your perceptions about how code is produced in large scale software development projects. This is a result of a fundamental requirement of transparency and communication which results in a development process that is distributed, extremely fast, and modular.

This article will explore how development occurs in an open source community. It will also explain how a typical open source community operates in order to provide context for how the actual development of code is carried out.

General Community Practices

Open source development is a highly collaborative process, and the only way for this to be successful is for all participants to make their technical motivations, intentions, and plans related to their participation visible to the rest of the community. This encourages greater collaboration and cooperative planning, which improves overall productivity of all parties and helps dependencies get resolved more quickly. It also results in greatly enhanced documentation because discussions themselves can be used as documentation and discussion archives provide important historical context for project decisions.

Public discussion is a prerequisite to all development in an open source community because it makes maintainers aware of the needs and problems community members have, and allows others to provide feedback and join the effort if their motivations are aligned with it. This helps keep various stakeholders on the same page to maintain the overall project direction. The decentralized decision making process that results from this has an added benefit of drastically improving the quality of the code.

Transparent Peer Review to Enforce Strict Quality Standards

Peer review is at the center of the decision making process in an open source community; this involves public discussions where anyone can comment on potential changes. Then feedback is offered primarily from the individuals who the change most impacts. Peer review helps maintainers process larger amounts of code faster by relying on trusted members of the community to enforce the strict quality standards the community has adopted. Feedback from the community can include commentary on code quality or formatting, the number of changes being made in a single submission, the degree of change the patch introduces, compliance with the overall strategy of the project or subsystem, or many other things.

Full Stack Involvement, From User to Developer

In a traditional corporate development environment, the development team is the only group that has access to resources like source code, requirements lists, documentation, and bug lists; users often have very little involvement in development before release.

In an open source project, development resources are made available to everyone, even those who are outside the development team. This means that source code and project requirements are published publicly, and bug tracking is openly searchable by anyone. Users and beta testers are also encouraged to submit bug reports. This tighter integration of users and developers results in better informed requirements before release, and it empowers users to find, report, and help fix bugs and other issues that are outside official test cases.

Decentralized Decision Making

Open source project leaders appoint delegates to make decisions for the community; these delegates build trust by being good participants in the community and by making good decisions on related issues. Transparency is vital because it is only possible to build trust by participating in public discussions, and often the discussions, themselves, are the documented record of the project direction. Anyone who wants to influence community decisions absolutely must participate on public discussion platforms like mailing lists, IRC, issue trackers, and project forums. With that said, not all decisions need multiple levels of review, and often trivial fixes are passed directly to the project maintainer.

How Open Source Compares to Proprietary Software Development

Proprietary development typically involves very little interaction between the developer team and the client that receives the software. If the client is commissioning the software development, their interactions are often limited to submitting a list of requirements to the developers, who then build the software behind closed doors. The client doesn’t see what is happening in the development process until the vendor is ready to deliver the next incremental version. If the client is simply a user of the proprietary software, their interactions are often more limited to communication channels such as support lines and premium consultation services.

In an open source community, users play an active role in proposing new ideas for the project and they can follow the entire development process from start to finish through public channels of communication. Anyone can submit patches, including users, and both developers and users are involved in the testing process. The roles of developers and users are much more closely integrated in open source development; this allows users to have a more direct path to influencing project development. This model also makes it much easier to integrate new features, and allows developers to respond to fringe test cases much more quickly.

One important consideration here is that while a vendor might be contractually required to fulfill their client’s requirements in the proprietary model, developers have no obligation to build user-requested features in an open source project. This can be remedied by hiring developers to add the features to the open source project or by finding active developers who share similar motivations.

The Development Process

Release Early, Release Often, and Test Continuously

One of the fundamental concepts of open source development is to release early, release often. Releasing early allows others to participate in development and provide feedback, making it easier for new ideas to be worked in while the code is still new and malleable, and allowing potential problems to be identified early.

Releasing often results in smaller changes that are easier to understand, debug, and mature, and it helps maintain the rapid development and innovation pace that is commonplace in open source development by giving all contributors the latest code on a regular basis. However, quick releases may lead to release fatigue in users that are forced to constantly update their system as the project progresses. To combat this, many projects release multiple versions of their code:

Long Term Stable (LTS) – This is for users that value stability above all else; LTS releases are typically supported for multiple years and typically only receive essential security updates.

Stable – This is for users and developers that want newer features, but only after developers users have tested them.

Experimental – This is the absolute newest code and has a high likelihood of containing bugs. This is typically only for developers who are looking to contribute to the upstream project, or who are building products that need to use features which are still under development.

Continuous testing allows open source communities to discover and triage problems more quickly, resulting in faster fixes. Since changes are typically smaller (release often), it makes troubleshooting much easier, and results in regressions being noticed earlier. This makes the work of maintainers to determine which code should be accepted easier. Open source communities often have multiple build cycles that overlap each other and align with release schedules and feature freezes. Automated build tools are often used for this process.

Modular Development

Staffing in an open source community is much more fluid than traditional software development communities. In a traditional software development model, developers are assigned to specific projects and typically don’t have much incentive to contribute to other areas. In an open source community projects will have clear owners, however developers are often encouraged to contribute to other projects or subsystems. They often have incentives to do so because the requirements of multiple projects can often align. Developers might have a primary project they work on, but it is likely they will contribute to a broad range or projects and subsystems.

As a result, open source code is often designed in an extensible, modular fashion to make it easier for individuals to contribute across subsystems. Where extensibility is often considered a feature in traditional software development it is a requirement in open source because it helps make incremental changes, follow an early and often release cycle, improve code maturity, and reduce overhead. A smaller core with features implemented as plugins reduces collisions and creates a natural separation of tasks and scope. This allows features to be made available to developers more quickly since the number of interdependencies is reduced and features can be released as they are completed.

Coming Up Next: The Tools of an Open Source Community

The most important points to take from this article is that transparent collaboration is an absolute requirement in any open source community, and one must interact with the community based on their established processes for doing so. As a result of this requirement, open source communities have coalesced around a relatively standardized set of tools that are used to facilitate transparent collaboration. The next article in this series will discuss the tools that are typically used in open source communities and what their roles are in the development process.

Share this:

Author: Ben Lloyd Pearson

Ben is an open source technologist experienced in a wide array of modern IT tools. He's done a bit of everything, from web development, to systems administration, web publishing, and beyond.
View all posts by Ben Lloyd Pearson