Reuse Through Internal Open Source

I haven't found an organization yet that doesn't want to foster reuse in its software development projects, thereby reducing development and maintenance costs, increasing quality and improving time to market. Unfortunately, most struggle to achieve this goal, even though a vast source of reuse is available only a mouse click away in the form of open source projects. These collaborative, public efforts, in which people improve the original software, adapt it for different environments, fix bugs and share their modifications with the rest of the world are, in many ways, the epitome of code reuse on an international scale.

A great place to learn about open source is OpenSource.org (www.opensource.org), a nonprofit organization dedicated to supporting and evolving the open source movement. CollabNet (www.collab.net) also provides tools and services based on open source tools and principles, and SourceForge (sourceforge.net) is a free open source hosting service. A quick search on the Web will find other organizations that offer a wealth of information, tools and services. The book Open Sources: Voices from the Open Source Revolution, edited by Chris DiBona, Sam Ockman and Mark Stone, (O'Reilly Publishing, 1999) is worth reading; in the spirit of the open source community, it is also published online in its entirety at www.oreilly.com/catalog/opensources/book/toc.html.

An interesting example of an open source project is Osage: Persistence Plus XML (osage.sourceforge.net). Osage is a persistence layer written in Java, supporting JDBC-based object-relational mapping to enable experienced Java developers to quickly implement database access in their applications. It generates SQL for basic CRUD (create, retrieve, update and delete) operations, maps Java objects into a relational database, maps object relationships (associations) and generates persistent object identifiers (keys). The design of Osage, in part, is based on my articles about persistence layers, many of which have appeared in Software Development over the years and have now been reprinted in The Unified Process Construction Phase (CMP Books, 2000), a book edited by Larry Constantine and myself. I have had little input into Osage other than that due to lack of time on my part, but I've been very impressed with the project to date.

Joining the Effort
So how can your organization benefit from the open source movement? First, investigate the software that is available, evaluate it against your needs, and reuse whatever fits best. Infrastructure software, such as Osage, is often immediately usable by a wide range of companies because it addresses a common technical need, in this case, the persistence of Java objects into a relational database. However, do exercise care with open source software. As Bertrand Meyer reveals in "The Ethics of Free Software" (Software Development, March 2000), you could be burned waiting for fixes or changes that don't materialize in a timely manner. Organizations that do decide to reuse existing open source software should be prepared to share any changes that they make by submitting them back to the projectopen source is a give-and-take proposition.

The second way you can benefit from the open source movement is to start a project of your own. Perhaps your organization requires software that you believe could fill a popular need, and after looking into both commercially and freely available software, you discover that you're the first to have thought of it. You start building the software and quickly realize that, while you can develop basic functionality quickly, some of the more complex functionality will incur a significant investment to both develop and maintain. You may decide to place your existing work in the public domain as an open source project, perhaps hosted by a company such as SourceForge, in the hope that others with similar needs will get involved and extend your work.

Third, you could start an internal open source project within your organization by setting up a private open source environment within your own intranet, as opposed to a public one on the Internet. You would make your work available to others within your organization, allowing them to evolve it and then share their changes as they see fit. This is true, honest teamwork between developersother project teams become involved if it is to their advantage, not because they are forced to by senior management. Your work must have an intrinsic value to the developers, however, by solving a problem that others are struggling with and doing so with sufficient quality. A properly run internal open source project allows you to achieve reuse within your organization, reducing your development and maintenance costs, while protecting your unique software from the prying eyes of the outside world, thereby safeguarding your intellectual property.

Indeed, internal open source, when implemented properly, reflects many common reuse practices (as I described in Thinking Objectively, "Reuse Patterns and Antipatterns," Feb. 2000). To throw some pattern and antipattern names around, internal open source results in the development of robust artifacts through self-motivated generalization while supporting and enhancing a reuse-friendly culture. It goes beyond traditional approaches to reusesuch as project-driven reuse, repository-driven reuse or items being declared reusableapproaches that often flounder because they don't motivate developers. Why do I believe that internal open source is attractive to developers? Most existing open source projects are efforts that developers work on in their spare timeif developers are willing to invest their own time working on open source software (OSS) then it isn't much of stretch to think that they would be willing to invest company time on an internal project.

Managing Internal Open Source
How do you run an internal open source project? Start by identifying a vision and general architecture for your efforts and then build an initial version of the software following your typical development process. This initial version need not be complete, but it should provide some initial value to the people who download it, while reflecting the overall vision so as to provide guidance to developers who choose to evolve it. A project page, or set of pages for large efforts, that links to your project's artifacts (for example, the project overview, the source code, frequently asked questions and licensing definitions) for the software, should be set up on your intranet. Ideally, the definition of usage should be fairly straightforward because your software should only be used within your organization, although issues such as chargebacks (and politics in general) can get in the way. And if you're bogged down defining your own internal usage license, your organization's culture likely won't allow you to successfully develop internal open source softwareor any other software.

Identify a project or team leader who will be responsible for shepherding the effort as well as accepting submitted changes. Shepherding often simply entails staying actively involved with the project's newsgroup, discussing suggested solutions and their potential impacts, communicating the vision, and trying to put people together who are working on similar issues. Changes need to be reviewed for applicability as they are submitted and then should be made available to the overall project. Larger projects will have hierarchies of people involved: Some will be allowed to directly submit their changes, whereas others, often people new to the effort, will first submit their changes through some sort of quality gate.

Contrary to popular belief and to their commonly chaotic appearance, open source projects are often well organized. Although not directed at nonproprietary projects, I recommend Adaptive Software Development: A Collaborative Approach to Managing Complex Systems by James A. Highsmith III (Dorset House, 1999) for a fresh perspective on how to be successful at developing software.

What tools do you need to make an internal open source project successful? Luckily, not many. You'll need a code repository, typically a version-control tool such as CVS (which is itself OSS) or Microsoft's SourceSafe, from which people may check out and submit code and other project artifacts. To facilitate communication between developers, you will need a newsgroup and/or a mailing list. Defect-tracking software enables developers to post identified problems and their solutions. Your organization likely has all of this software in place already; however, any of it can easily be obtained at no cost over the Web (once again, as OSS) if needed.

Making It Work
To ensure success, first inform developers that the software is available for their use either through an internal marketing campaign or by word of mouth. Second, apply internal open source not only to code but also to documentation templates, internal standards and guidelines, and common software models. Third, focus on quality and not quantitytwo or three reusable programs employed by several projects offer more value than 20 or 30 projects gathering electronic dust. Fourth, overcome cultural and political obstacles. Forget cost chargeback schemes, forget complex management processes, forget endless meetings and instead focus on developing software in a collaborative manner. Fifth, be prepared to keep an eye on individual developers: Programmers should be plugging away at internal open source because it makes sense to do so for their current, primary project, not simply because they want to ignore their assigned work.