If this is your first visit, be sure to
check out the FAQ by clicking the
link above. You may have to register
before you can post: click the register link above to proceed. To start viewing messages,
select the forum that you want to visit from the selection below.

Comment

They aren't contributing back, this is their own incarnation of a test infrastructure from their OSRC. Doesn't make too much sense to me, but oh well.

Yeah you are right, contributing back would mean pushing some improvements back to Phoronix. Releasing their code is a first step, and I was referring to that, but they could become a downstream product, keep in sync, and work with you. That would be the best approach IMHO. Not only it would benefit Phoronix, it would benefit AMD, because they's inherit all your improvements all the time. This is he beauty of Open Source

Comment

Yeah you are right, contributing back would mean pushing some improvements back to Phoronix. Releasing their code is a first step, and I was referring to that, but they could become a downstream product, keep in sync, and work with you. That would be the best approach IMHO. Not only it would benefit Phoronix, it would benefit AMD, because they's inherit all your improvements all the time. This is he beauty of Open Source

Unless I misread something, Tapper is not in any way based on PTS... it's a completely separate project created by AMD.

Comment

Tapper is exactly not yet another test or benchmark suite. It is atest infrastructure which among other things allows to write or
embed test suites.

In that the focus is much larger, much more targeting an enterprise QA
life cycle than just being a test suite.

Tapper allows gluing together components based on existing
standards. One central idea for that is the standard Test Anything
Protocol (http://testanything.org) which allows language agnostic
generation of test results.

In order to run a Tapper client it is not neccessary to have a
Tapper server. You can run a TAP emitting testsuite with the standard
command "prove" that comes with every Linux system, like

Code:

prove ./tap-emitting-testsuite.sh

Only when you want to store your results into the history database you
need a server. And then it's nothing more than just

Code:

./tap-emitting-testsuite.sh | netcat $tapperserver $port

An important focus left out in the post is the advanced result evaluation.
Please see this presentation
to understand how any complex results, that are even allowed to
structurally change over time, can be tracked and evaluated.

It provides a query language that combines a database independent SQL
abstraction and a variant of XPath all in one concise API without the
need of client side dependencies. The point of that is to allow test result forensics in ways you don't know in advance.

The opensourced version already ships with a wrapper for the autotest
client - that immediately provides you access to about 90(!) test
suites. For that the OSRC contributed TAP support to the autotest
project. Seeautotest mailing list for details.

And Tapper is actually used with a Phoronix test suite wrapper at the
OSRC. For phoronix it would be most practical to be embedded into
autotest to be useful to more test communities. The Tapper-TestSuite-AutoTest
would then just pick it up automatically.

Tapper only implements features that were not available in 2008 when
they were needed, like virtualization setups for stress testing
(inclusive windows test suites in the guests), or when available
solutions were not enough, like the scheduler.

The automation layer provides a built-in installer to set up
virtualization scenarios but others can be plugged in easily. That is
actually done for kickstart/autoyast based setups and for simnow
testing.

On top of that the scheduler allows multiplexing machine pools with
any freely definable use-cases. It is built to schedule "too many
use-cases" on "not enough machines" based on bandwidths.

Setting up a Tapper instance for tracking test results is easy as all
dependencies are resolved by standard CPAN installation. The
non-trivial setup of an automation infrastructure with virtualization
support is additionally supported with the Tapper-Starterkit.

To complete a QA life cycle Tapper can schedule dedicated testplans
that belong to tasks defined by a planning software like
TaskJuggler. The results are reported back to TaskJuggler and
therefore covered in the project status pages.

The layers are strongly decoupled and all interfaces are very
lightweight, to allow glueing more things together in the future.