Revision as of 21:25, 9 February 2009

Many minds met during FUDCon F11 Boston 2009 in order to brain storm on automated testing.

We identified a number of "triggers" to react to in order to test things, a number of tests to perform, and a set of places to notify of the results. Here is a quick dump of these three things.

Triggers

cvs checkin

koji build

bodhi request

repo compose

install tree compose

manual (periodic)

iso compose

new test creation

post test

comps change

Tests

repository Sanity (sublist here)

package installation

profile testing

tree sanity

rpmdiff

build log sanity

source sanity

build root sanity

fails to build from source

make check

comps grammar

Notifiers

<package>-owner@fedoraproject.org

project webpage

fedora-test-list@redhat.com or fedora-devel-list@redhat.com

Also discussed were timeline targets. Next 3 weeks, next 3 months, next 3 quarters. We feel we can accomplish a number of the triggers and tests within the next 3 weeks, focusing more package and repo sanity. In the next 3 months we can likely add functional testing of installer via Labin aBox. Over the next 3 quarters we hope to be able to expand functional testing to individual packages.