Cool hack: annotating compatibility in test suites

Here's a fun bit of code that I didn't really expect to work. It flags a test as TODO only if client code indicates that it's designed for an older version of a test suite.

Background: I'm working on Test::Virtual::Filesystem, which is a collection of generic tests that exercise a filesystem with typical actions like open, read, write, stat, unlink, mkdir, etc. Users of this code will run it in a.t file to ensure that their filesystem works. But what if I change Test::Virtual::Filesystem to add a new test that breaks some filesystem published on CPAN that used to work with the old version of the test suite?

To solve this, Test::Virtual::Filesystem is written like so (trimmed heavily for readability):

The Fine Print: The following comments are owned by whoever posted them. We are not responsible for them in any way.
Without JavaScript enabled, you might want to
use the classic discussion system instead. If you login, you can remember this preference.

Please Log In to Continue

This is a very interesting little hack. We did something similar at one job where we added a 'DB' attribute to our tests and our test database would only be built if said attribute was found. Nifty little speedup. I love seeing more cool stuff for my favorite testing module.

I added another one after writing that blog. The version headed to CPAN right now has a:Features('foo,bar') attribute to let clients turn on and off swaths of tests. For example, 'symlink' is one of the features, 'xattr' is another. If a feature is not enabled, I call skip on the method, and Test::Class offers a nice hook to fetch the number of tests to skip.