Summary

A web-based test reporting system for release testing of ISO images.

Release Note

Contributing to the CD testing effort in Ubuntu has been made easier with a purpose-built test tracker.

Rationale

For the Feisty release process we used first an improvised system in Malone and then a prototype version of a dedicated web-based tracker. This system should be matured and extended to become an efficient and reliable tool in the release process.

Use Cases

Community tester

Release Manager

Test admin

Assumptions

Design

New features

Tabbed browsing: Organise the different distro flavours by tabs to reduce the length of the ISO list [DONE]

Trusted submitters: Create a category for users that are known to produce reliable test results. A pass or fail report from a trusted user will propagate up to the test and ISO summaries, reducing work for the administrators. Alternative[DONE] (Test/Build status removed, pass/fail on results added and subscription now possible)

Bulk email: New registrants should receive an email with a link to on-line documentation of how the system works. Allow the administrators to send email notifications to registered testers about new ISO images, etc. Allow users to opt-out of receiving messages. [DONE]

Gather Launchpad bug info: Hovering over a bug should show its title. The status of a bug should also be reflected on the test site: Is it confirmed, serious, milestoned, etc.? [DONE]

Automation of test posting: Connect the script used to generate the ISO images with the tracker so that building milestone test images also posts the tracker entries. Realistically - add a single command line step to post the test entries after the builds are done. [IN PROGRESS]

Improve navigation efficiency: Explore ways to reduce the number of page levels. Most likely make the test cases expandable so that the list of submitted tests can appear below it (click to show and hide). [DONE] (archives and my subscriptions are done that way now)

Archives: As we grow more archive information we need to organise it better, probably by milestone. [DONE]

Look and feel: Create a theme that is attractive and efficient. Add the new testing logo, use icons that are easier to spot and use the screen space more efficiently. [DONE]

Download link: Provide an easy link to download each ISO image. Also provide md5sums and a sample rsync command that can be used. [DONE]

Add checkbox for the release team to set a bug as a blocker.

Report generation to show which bugs have been fixed for the release and which are still open issues. [DONE]