You can find quite an interesting panoply of open source operating systems to choose from. Being open source you can have access to a lot of their code, build systems, bug tracking, etc.

As of this time I didn't came across a document that plans or shows the design of stability testing for an OS. It would be useful for me to know about: what test suite they use, what benchmarks, what stress tests, do they have some stability test plans released to the public, etc?

1 Answer
1

Short version

Not necessarily.

Long version

The idea

First thing to realize is what open-source means for software: the basic idea is that the source code is available. In more specific terms, someone has decided (for a new project or a running one) to share the code, i.e. every technical detail with the whole world. Furthermore, since there's technically nothing that can stop people from compiling and/or fully using such code it typically also means that the software is free (as in free beer) to use.

That's about the idea of open source. It has nothing to do with planning or organization of human resources.

The community

So in practice, and if the project is successful and interesting enough, the above will also mean that there will be people who will actually (try to) use it, and there will be people that will look at that code, study it, analyse it, play with it and talk about it, and eventually send bugfixes back to the author.

From this list of activities, at least three involve, or are a form of what is known as software testing.

So vaguely said, the quality in FOSS comes from:

lot of discussion before design and coding

perpetual code peer review

feedback from people trying to play with the the SW

feedback from people trying to use the SW in real situations

feedback from people trying to help develop the SW (maybe just fix their
use case)

sometimes even feedback from professionals paid by companies that have
their business built on FOSS technologies

no deadline tyranny

and also quality of the original code base and architecture

much more than creation and execution of testing plans (perfectly ballanced to match the needs of the focus group).

The strategy

Other thing to consider is, that each planning must start with a strategy. For commercial products, strategy is money and money is strategy: Who will run it? How will they run it? How much will get back at us? How much support requests we'll be able to handle? How many different hardware platforms even exist?

The strategy part is really a tough call since it usually happens before anything else, and the decisions taken here will influence almost every aspect of the whole project, and are extremely expensive to change. Therefore great amount of information needs to be gathered, which costs additional resources.

No need to say that any reasonable test planning needs to copy the strategy. (You're not going to test the brand new .NET-based accounting software with Powershell-powered installation routines on Win 2000, are you?)

Since most open source starts either as "scratching someone's own itch", without much focus on how much other people have it, it's virtually impossible to take this step, but it's also much less important, since more users can't scratch itch any better than the developer can themselves. If the scratcher is not sharp enough, they can help sharpening (send a patch).

The conclusion

Just in case you got that feeling: I'm not trying to say that there cannot be a highly organized group of professionals proceeding with a rigorous and well-thought testing plan on an open source. It definitely can and many such teams definitely do exist (I think Red Hat might have group like that for Fedora). But just because they are open-sourcing the code, it does not mean that they are willing to open-source their know-how.

At the end of the day, what counts most are bug reports, test cases and patches.

More specific to the question: for an open-source OS, there isn't a specific thing called a stability test, but lots of people (with a vested interest in stability) test it in their own environment.
–
user246Jul 10 '13 at 0:37

@user246 Can you give a link with an example of other organization running some stability tests on some open-source OSs?
–
Eduard FlorinescuJul 10 '13 at 6:19