The ‘ratio’ is the number of bugs per test. Obviously there’s wiggle room here; different people report different bugs, and some of the drivers implement features the others don’t (and hence have more ‘surface area’ for bugs). But I think they’re quite fun anyway. Nouveau and Radeon both regressed from f11 to f12, according to the numbers, and got better than either previous release for f13 (at Test Day time). Intel has stayed fairly steady. According to this analysis, Nouveau wins the ‘least buggy driver’ contest by a fair margin, which is interesting! For F13, it has half as many bugs per test as Radeon, and a third as many as Intel. (Of course, I’m sure some of our erstwhile devs would argue it could fairly be renamed the ‘least crack-addled hardware manufacturer contest’…)

The F11 Test Days happened in March 2009, the F12 Test Days in September 2009, and the F13 last week; these are pretty comparable points in the respective cycles.

In terms of participation, we had the largest number of testers for F11, with the nouveau number accounting for most of that; I suspect this is because nouveau was very new and shiny in F11 and impossible to use on most distros, so people were very interested in trying it out for the first time. F12 had the fewest tests run, and F13 pretty much splits the difference.

I counted CANTFIX, WONTFIX and INSUFFICIENT_DATA as ‘unfixed’, ERRATA, RAWHIDE, CURRENTRELEASE and NEXTRELEASE as ‘fixed’. NOTABUG I lumped in with DUPLICATE as the ‘closeddupe’ number (these are reports that should be discarded from consideration entirely). The percentage is calculated as:

closedfixed / (bugs – closeddupe) * 100

i.e. it roughly indicates the percentage of genuine unique bugs reported that have been fixed so far. These numbers are pretty close, both across drivers and across releases; we’ve fixed just over half the bugs reported. I think the outlying high nouveau result is probably a consequence of ‘low-hanging fruit’ – the driver was in a pretty initial state at that point, so the bugs exposed are likely to have been, on the whole, easier to fix. Obviously I’ve left F13 out as the maintainers have had only half a week to work on the bugs!

Thanks very much to all testers, and to the wonderful Fedora X.org developers and triagers: