Open Source releasing best practices?

Hello,

When creating open source software, I don't really know what the best way to release it (what kind of makefile, versioning system, etc...) is. It's also hard to find this info online, is there any good online info or book about this?

Here are some of my questions...

-How many operating systems should your makefile support? Should I make special cases for every single Linux distro and other OS in the makefile, or can I just put a generic "g++ *.cpp" in the makefile, and let each OS and distro's own package managers take care of tailoring it to their OS?

-For makefile complexity, I guess there is a scale ranging from a hack like just typing "g++ *.cpp" in it, through having nice sections, groups of files and definitions like "CFLAGS", all the way up to projects which have 20 different makefiles in them like "Makefile.in", "Makefile.pandora", etc.... Where on that scale should you ideally be?

-Makefiles of many projects look incredibly complex, why?

-What versioning system to use? When to make a 1.0.0? When to append "rc" at the end?

-When to tag stable versions? And when you change something in head, do you need to change version number every single time?

-When creating a dynamic library, and you tagged a stable version, and you then change something in head. Should in the makefile the version number of the library name be changed to something? If so, should it be changed to a next minor version, or to something with "-rc" at the end?

-What names should be used for tags of versions?

-Does there need to be both a zipped version of the source code and one under VCS, and if so why is that zipped version needed?

-Are there any naming conventions for output binaries and libraries?

-Are you supposed to let your makefile clean up .o files after compilation or not?

-Are there any conventions for makefiles for names of sections and variables in it? E.g. is it a good idea to have a "clean:" in your makefile to remove everything?

-When depending on another library which is hosted somewhere else, how to handle that? What when statically depending on it?

Re: Open Source releasing best practices?

Wow, that is a lot of questions Let me answer some of them. Note that all of the answers will reflect my personal views and opinions, I do not claim them to be 100% correct or universally usable.

aardwolf wrote:

-How many operating systems should your makefile support? Should I make special cases for every single Linux distro and other OS in the makefile, or can I just put a generic "g++ *.cpp" in the makefile, and let each OS and distro's own package managers take care of tailoring it to their OS?

Ideally, the makefile should be universal. When a packager for certain distro needs it, he can always modify what is necessary. As long as you comply with FSH standards and common makefile conventions (more on that below), there should be very little work left to do for the packager.

aardwolf wrote:

-For makefile complexity, I guess there is a scale ranging from a hack like just typing "g++ *.cpp" in it, through having nice sections, groups of files and definitions like "CFLAGS", all the way up to projects which have 20 different makefiles in them like "Makefile.in", "Makefile.pandora", etc.... Where on that scale should you ideally be?

Either the project is complex, or the people just like complicated things

aardwolf wrote:

-What versioning system to use? When to make a 1.0.0? When to append "rc" at the end?

1.0.0 should be first stable release, after which you don't expect major changes in funcionality. Other than that, there is not much rules to abide, just keep the numbers going up. Use common sense to decide whether the change is just bugfix or whether it deserves major or minor version increase.

aardwolf wrote:

-When to tag stable versions? And when you change something in head, do you need to change version number every single time?

When the code is stable The number should change when you release the code to public, e.g. when a new packages are published. Code in HEAD revision of repository doesn't need to be reversioned after each commit.

aardwolf wrote:

-When creating a dynamic library, and you tagged a stable version, and you then change something in head. Should in the makefile the version number of the library name be changed to something? If so, should it be changed to a next minor version, or to something with "-rc" at the end?

Until you publish it officially, it can use whatever version suits you

aardwolf wrote:

-Does there need to be both a zipped version of the source code and one under VCS, and if so why is that zipped version needed?

The compressed archive is often used for packaging purposes, it is easier to download it and create a package (PKGBUILD, deb, rpm or whatever) then pulling code from VCS. So there should be tarball for each released version of the code.

aardwolf wrote:

-Are there any naming conventions for output binaries and libraries?

Nothing special that I'm aware of, apart from common stuff like no weird characters (whitespace, non-latin etc.)

aardwolf wrote:

-Are you supposed to let your makefile clean up .o files after compilation or not?

No, .o and other intermediate files should be only cleaned in clean: target. It is the primary purpose of make, to avoid unnecessary recompilation, unless some files changed.

aardwolf wrote:

-Are there any conventions for makefiles for names of sections and variables in it? E.g. is it a good idea to have a "clean:" in your makefile to remove everything?

The most common targets are 'all' (usually default target), 'test', 'install', 'clean' and sometimes there is also 'dist-clean'. It is definitely good idea to keep those, as many people try to use them automatically, and only if it fails, they look into documentation

aardwolf wrote:

-When depending on another library which is hosted somewhere else, how to handle that? What when statically depending on it?

Dynamic dependencies are handled by package management software, you don't need to worry about that. Static dependencies can be sometime handled through packaging too, if the system supports it (something like makedepends in PKGBUILD). In both cases, you should mention the dependencies in documentation.

Re: Open Source releasing best practices?

Some of what you are asking may be dependent on the licence you choose to distribute your code.Note that, if you are using licensed code, you must be in compliance with whatever licence grants you that right. If, for example, you are using the GPL, you must provide the information for how to compile and link the source. Often, that is handled by providing a well designed Makefile or (in my opinion, the better choice) the input files to drive Autotools.

Rules vary globally as to default copyright rules, so pick a license that will enable others to redistribute your work (if that is your intention) while preserving your right. Personally, I like GPL-2 for most of my projects, YMMV. Ensure sure you put your name, date, contact information and copyright declaration in all of your source files. Refer to your chosen license for details.

Nothing is too wonderful to be true, if it be consistent with the laws of nature -- Michael FaradaySometimes it is the people no one can imagine anything of who do the things no one can imagine. -- Alan Turing---How to Ask Questions the Smart Way

Re: Open Source releasing best practices?

The Makefiles that you almost always see are created by GNU Autotools and not written by hand. GNU Autotools can be very complicated for the developer but makes things very simple for the end users and packagers of different operating systems.

I actually got into trouble by not using GNU Autotools. Some nice video game reviewer found my game on SourceForge, downloaded it, and failed to compile it. His review would have said "This game didn't work" if it wasn't for another nice person pointing out that he needed another package installed, after which my game worked fine (and got a fine review ). If I had used GNU Autotools, then he would have easily known what the problem was in the first place.

Re: Open Source releasing best practices?

Re: Open Source releasing best practices?

aardwolf wrote:

Hello,

Many of your questions have no single correct response. I'll reply with my own opinions and experiences, based on my release of two open source projects (GPT fdisk and rEFInd).

-How many operating systems should your makefile support? Should I make special cases for every single Linux distro and other OS in the makefile, or can I just put a generic "g++ *.cpp" in the makefile, and let each OS and distro's own package managers take care of tailoring it to their OS?

Ideally, a Makefile should build a package under every OS on the planet. In practice, this isn't always practical. Many developers use programs like Autotools to create Makefiles that are suited to a particular build environment. Other developers (myself included) create a handful of Makefiles for different environments -- for instance, my GPT fdisk has Makefiles for Linux, FreeBSD, OS X, and Windows. My rEFInd officially supports building only under Linux, although it supports two EFI toolkits (GNU-EFI and TianoCore EDK II) via a cascading set of Makefiles. Any of these Makefiles can require changes depending on the distribution and development environment in use, but that's not really my concern.

If a distribution requires changes, that type of change is generally best left to a build system like Autotools or to the person who builds or packages the program. IMHO, it's unreasonable to ask a developer to make minor tweaks to a static Makefile to support every minor Linux variant on the planet.

-For makefile complexity, I guess there is a scale ranging from a hack like just typing "g++ *.cpp" in it, through having nice sections, groups of files and definitions like "CFLAGS", all the way up to projects which have 20 different makefiles in them like "Makefile.in", "Makefile.pandora", etc.... Where on that scale should you ideally be?

This is very much a matter of personal preference and project complexity. Autotools or something similar will make it easy for users and distribution maintainers, but can be tricky to use for the developer. If your program is a simple single-file C program, you might forego a Makefile completely; but for something on the scale of the Linux kernel, a Makefile (or something equivalent) is absolutely required.

-Makefiles of many projects look incredibly complex, why?

Some projects are very complex, as in the Linux kernel itself. Other times, the Makefiles generated by automated systems like Autotools can be more complex than they might be if they were hand-crafted. In still other cases the developers like complexity or are barely competent at creating Makefiles and so create something that's more complex than it needs to be.

-What versioning system to use? When to make a 1.0.0? When to append "rc" at the end?

AFAIK, there are no standards on this. A 1.0 release denotes that something has moved beyond "beta test" status -- in other words, you think it's stable and usable for the masses. Open source software authors tend to be conservative in making that judgment, so pre-1.0 releases in the open source world are often as good as post-1.0 releases of commercial software. The bottom line, though, is that it is a judgment call -- what I consider "1.0" software you might consider well beyond that point and something else might consider pre-beta.

As to release candidate (RC), not all projects use that designation at all. It seems to me to be more common among large projects as they approach major release milestones, to denote something that is close to being finalized, but not quite -- essentially a sort of very late beta stage, even if the initial 1.0 release was made some time before.

-When to tag stable versions? And when you change something in head, do you need to change version number every single time?

If the code changes, you should definitely change the version number. Most developers accumulate several changes before making a new official release, though. Personally, I make full releases with three-digit numbers (like 0.8.6 or 0.6.10), and I upload minor changes to my project's git repository with four-digit numbers (like 0.8.6.1 or 0.6.10.2), but don't do full releases with tarballs and RPMs and whatnot for these, except in a limited way if I want specific people to test a recent change because they filed a bug report. Others have other systems.

-When creating a dynamic library, and you tagged a stable version, and you then change something in head. Should in the makefile the version number of the library name be changed to something? If so, should it be changed to a next minor version, or to something with "-rc" at the end?

The key difference with dynamic libraries is that the interfaces should not change with minor changes. IIRC, the second digit (like "2" in 1.2.3) is the cutoff point. In other words, a program that uses library version 1.2.3 should continue to work without changes or recompilation with library 1.2.4 or 1.2.2 (assuming no bugs). This enables users to upgrade the library (from 1.2.3 to 1.2.4 or the like) without upgrading every binary that relies on it. With version 1.3.0, though, the interface to the library might change in a way that would require recompilation of the program or even changes to the source code. Thus, changing the library from 1.2.4 to 1.3.0 will require the user to upgrade all the programs that use that dynamic library (or keep the old version around along with the new one). Note that I've never created a publicly-released library, and it's been a while since I've read up on this, so I might be a little off on these details.

-What names should be used for tags of versions?

I'm not sure what you mean by this.

-Does there need to be both a zipped version of the source code and one under VCS, and if so why is that zipped version needed?

You can do it any way you want; but as a general rule, you should provide source code in a tarball or .zip file because that's easier to download. Some package systems, such as RPM, require that a source package filename be specified, and so not providing source in such a package just complicates matters for packagers and therefore makes it less likely that they'll bother packaging your program at all. This in turn makes it harder for your users to use the program.

Note that most Linux programs' source code is provided as tarballs rather than as .zip files. Some cross-platform programs can be exceptions to this rule. For instance, I used .zip for rEFInd (a boot loader) because .zip is a little more common in Windows -- although I'm sure either would have worked fine, in practice.

You should probably provide binary builds of your software -- although in some cases this can be tricky because a binary built for Distribution A may not work on Distribution B because of library differences. The OpenSUSE Build Service (OBS) can help with this, although it's a bit of a pain to use.

-Are there any naming conventions for output binaries and libraries?

Not AFAIK, except of course for filename extensions like .so and .a.

-Are you supposed to let your makefile clean up .o files after compilation or not?

No, except for the "clean" target and anything else that's supposed to do this.

-Are there any conventions for makefiles for names of sections and variables in it? E.g. is it a good idea to have a "clean:" in your makefile to remove everything?

The "all" target builds everything, "clean" cleans up, "install" installs everything, and "uninstall" uninstalls everything. There's no law that says you have to have all of these, but they're common, particularly with big projects.

-When depending on another library which is hosted somewhere else, how to handle that? What when statically depending on it?

This type of thing is generally handled by packaging programs (pacman, rpm, dpkg, etc.), not by developers' Makefiles. That said, Makefile builders like Autotools should check for the relevant development libraries and stop if they aren't present. That will handle the static linking issue, as well as other problems. On another level, when using RPM, a source RPM will include dependencies on the relevant development libraries, and Debian source files have a similar feature. Putting these files together is the responsibility of distribution maintainers, not of program authors.

-Any other things I should know?

There's a huge range of acceptable practices on these issues. As a general rule, though, the smaller the package the more likely you are to find a simple Makefile that builds the whole project. Bigger projects are more likely to rely on multiple Makefiles, Autotools, or other complex pre-build software. More standardization emerges at the distribution level, in the form of source and binary RPMs, Debian packages, etc. You shouldn't need to worry too much about that. So long as your package builds with few or no changes on a variety of distributions, the distribution packagers can handle the rest. Build systems always support patches so that minor changes to Makefiles or whatnot can be incorporated. This frees you up to worry about other things rather than trying to support every minor variant distribution in existence.

Re: Open Source releasing best practices?

ewaller wrote:

Some of what you are asking may be dependent on the licence you choose to distribute your code.Note that, if you are using licensed code, you must be in compliance with whatever licence grants you that right. If, for example, you are using the GPL, you must provide the information for how to compile and link the source. Often, that is handled by providing a well designed Makefile or (in my opinion, the better choice) the input files to drive Autotools.

Rules vary globally as to default copyright rules, so pick a license that will enable others to redistribute your work (if that is your intention) while preserving your right. Personally, I like GPL-2 for most of my projects, YMMV. Ensure sure you put your name, date, contact information and copyright declaration in all of your source files. Refer to your chosen license for details.

I think the meaning of this may be misinterpreted so I would like to clarify a few points.

You only need to respect the license of other people's code. For example, if your code is based on or includes someone else's GPL code then you must release your code under the GPL (or a compatible license). If the code is entirely your own, you can do whatever you want with it. A license is a statement of the conditions under which others may use your code. You can release one version under the GPL and then release the next version under a BSD license. You can withhold sources and any other information that you like. You cannot violate the terms of your own license if you are the copyright holder.

Once you release some code under a given license, however, you cannot change the license for that release. For example, if you release version x under the GPL, everyone can use version x under the GPL forever, including all GPL-compliant derivatives of that release. If you then release version x.1 under a different license then all new code in that release is protected and others would need to be compliant with the new license to use those changes, but the original code from version x remains under the GPL.

You can also release code under multiple licenses. For example, you can release version x under the GPL for the general public and then release it under an exclusive license for a company that wants to include the code in some commercial hardware.

The last thing to keep in mind is that if you include submitted (non-trivial) code from others then you may eventually be bound for all future releases by the license under which that code was submitted. For example, the Linux kernel can almost certainly never switch from the GPL to a different license because it is a giant patchwork of submitted code, almost all of which has been submitted under the GPL.

As for what constitutes "trivial" code, that is another discussion. Personally I think that most code is trivial and I would be against all forms of copyright altogether if it weren't for the fact that a complete lack of such would lead to massive exploitation by greedy, a-/immoral companies.

Re: Open Source releasing best practices?

You may also want to read the packaging guidelines (https://wiki.archlinux.org/index.php/Ar … _Standards) which gives an idea of what information is required. For example PKGBUILDs can get sources from a version control system or a tar.xz file so its your choice.

I suggest to make your makefile as generic as possible (e.g. usig GNU autotools). Makefiles are complex because compiling, linking, packaging, ... software is complex.

First goal might be to make your package installable with makepkg and pacman (this should also answer your dependency questions). Then improve.

Re: Open Source releasing best practices?

As for licensing: the best practice is to include a license statement at the top of every (nontrivial) source file. Something like this:

/*
* Copyright (c) YYYY YOUR NAME HERE <user@example.com>
*
* Permission to use, copy, modify, and distribute this software for any
* purpose with or without fee is hereby granted, provided that the above
* copyright notice and this permission notice appear in all copies.
*
* THE SOFTWARE IS PROVIDED "AS IS" AND THE AUTHOR DISCLAIMS ALL WARRANTIES
* WITH REGARD TO THIS SOFTWARE INCLUDING ALL IMPLIED WARRANTIES OF
* MERCHANTABILITY AND FITNESS. IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR
* ANY SPECIAL, DIRECT, INDIRECT, OR CONSEQUENTIAL DAMAGES OR ANY DAMAGES
* WHATSOEVER RESULTING FROM LOSS OF USE, DATA OR PROFITS, WHETHER IN AN
* ACTION OF CONTRACT, NEGLIGENCE OR OTHER TORTIOUS ACTION, ARISING OUT OF
* OR IN CONNECTION WITH THE USE OR PERFORMANCE OF THIS SOFTWARE.
*/

Or this:

/*
* Copyright (c) YYYY YOUR NAME HERE <user@example.com>
*
* This program is free software; you can redistribute it and/or modify
* it under the terms of the GNU General Public License as published by
* the Free Software Foundation; either version 2 of the License, or
* (at your option) any later version.
*
* This program is distributed in the hope that it will be useful,
* but WITHOUT ANY WARRANTY; without even the implied warranty of
* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
* GNU General Public License for more details.
*
* You should have received a copy of the GNU General Public License
* along with this program; if not, write to the Free Software
* Foundation, Inc., 675 Mass Ave, Cambridge, MA 02139, USA.
*/

It may seem redundant to have this in every file, but open source projects often bring in files from external projects, which necessarily have their own copyright statements… a single COPYING file in the main directory is not enough (though it helps). For GPL projects, this also makes it clear whether or not the “or any later version” exception applies, as the license itself does not contain that clause.

If there is no copyright statement in a file, it’s legally “all rights reserved” in most countries, and as a packager, it means that sometimes I can’t legally release a package for a particular program.