Self-Describing Tests

A self-describing test is a test page that describes what the page should look like when the test has passed. A human examining the test page can then determine from the description whether the test has passed or failed.

The following are some examples of self-describing tests, using common techniques to identify passes:

They can test areas of the spec that are not precise enough to be comparable to a reference rendering. (For example, underlining cannot be compared to a reference because the position and thickness of the underline is UA-dependent.)

Failures can (should) be easily determined by a human viewing the test without needing special tools.

They also have some disadvantages:

They cannot be automated: a human must determine whether the test has passed or failed.

In some cases it is difficult or impossible to design the test for a glaringly obvious pass or fail. (In these cases, if it's possible to create a reference, the reftest format may be more appropriate.)