I think we should just switch the default for text vs. pixel tests. Make
tests require a <script> layoutTestController.dumpPixels()</script> to dump
pixels.
We could also white-list certain directories.[1]
-eric
1. If we did add white-listing, I'd like to move away from our
hacky-in-run-webkit-tests white-lists to an explict on-disk per-directory
config file or similar. Understanding why "fast/loader" turns on the loader
callbacks when you're writing a test has driven more than one
would-be-webkit-hacker crazy. :)
On Wed, Dec 8, 2010 at 12:43 PM, Simon Fraser <simon.fraser at apple.com>wrote:
> On Dec 8, 2010, at 12:13 PM, Peter Kasting wrote:
>> If you never write layout tests, you can stop reading.
>> Right now few ports run pixel tests (a shame). One reason is because we
> frequently need different reference images for each port, and creating
> hundreds or thousands of these is a hassle. Maintenance burden also
> increases.
>> We could greatly decrease the number of these baselines by following a
> simple rule: don't display text unless you need to. For example, let's say
> I have a test that is supposed to display a green square. If the test
> output is:
>> A green square means this test passes. If there is any red below, fail.
> [green square here]
>> ...then for each platform that has different font rendering, whether that
> be subpixel AA differences, font size differences, or anything else, we'll
> need new baselines. By contrast, if that explanation was in an HTML comment
> inside the test code, and the output was a simple green square, one baseline
> would serve for all ports.
>> I think this doesn't really hinder tracking down test failures, for a
> couple of reasons:
> * Most tests are "green = pass, red = failure" by convention, so frequently
> a pixel result that differs from the baseline is easily classifiable at a
> quick glance.
> * When this isn't the case, one of the first steps in understanding the
> test output is to read the test, at which point the HTML comment will
> explain things.
>> Obviously, some tests need to display text, but this seems to me to be a
> good rule of thumb.
>> Am I missing anything? If not, is anyone interested in helping to make
> this change on existing tests where possible, to trim the number of
> baselines in the tree and make it easier for new ports to bring up pixel
> tests?
>>> I totally agree, and have been avoiding test in pixel tests as much as
> possible recently.
>> One useful strategy is to add explanatory notes to the test as HTML
> comments, so the purpose of the test is easily determined by someone looking
> at the source.
>> Simon
>>>> _______________________________________________
> webkit-dev mailing list
>webkit-dev at lists.webkit.org>http://lists.webkit.org/mailman/listinfo.cgi/webkit-dev>>-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.webkit.org/pipermail/webkit-dev/attachments/20101208/1698186f/attachment.html>