Like this:

Related

One Comment

Much nicer! I really like having the raw numbers there: they give a good overall sense of what you are doing, they allow me to spot check and make sure my understanding of how the graphs are constructed is correct, and they should allow for rough comparisons between different sets of numbers you post or with numbers people come up with themselves. (Obviously different sets of code being tested can make that deceptive, but taken with the right caveats, it’s very useful.)

The absolute times graph corrects some of the defects of the times-faster times-slower graph (it’s easier to understand, it doesn’t spend half the space just to describe one single data point – the firefox-talos-gfx number for drm – even if that number is impressive.) But it has some problems of it’s own: it implies comparability between the different absolute times, and the test-sequences with small absolute times are compressed to illegibility.

I fooled around with this a bit (another advantage of posting the raw numbers), and came up with:

[ Yeah, the legend is in backwards order: a gnumeric bug apparently ]

Advantages I see: using the simple time/image_time metric puts all the tests runs on an equal footing, but is easier to understand than times-faster-times-slower, and preserves the ability to visually compare pairs of numbers (how do the xlib/xvfb backends compare?). The horizontal bars avoid vertical text. Truncating the x axis at 4 avoided compressing everything down to fit in one outlier number.