All the Perl that's Practical to Extract and Report

Navigation

The Fine Print: The following comments are owned by whoever posted them. We are not responsible for them in any way.
Without JavaScript enabled, you might want to
use the classic discussion system instead. If you login, you can remember this preference.

This is a great read, as well as your last journal entry on this topic. I'm in a similar situation, and have come up with a couple of ideas on tackling the huge codebase with failing tests monster.

First is to chop off a piece of the application and roll it into it's own self contained module. Now this is undoubtedly easier said than done, but if you can make that happen, you've downsized the overall problem a bit, plus the it's much easier to run the tests for that module when work needs to be done on it.

Second is to introduce a verbose (there's probably a better word for this) testing mode that is set with an environment or package variable. Some of our tests are essentially smoke tests which can take several minutes to run. They are a huge pain to run at checkin time, so on some of them we do skip if $ENV{TEST_LARGE}. This allows the majority of the unit tests to be run at checkin time, and then (theoretically) the automated build runs the large tests and reports failures there. It's far from perfect, but it takes care of a lot of cases where the small tests wouldn't be run anyway (the really slow case)

I guess a lot of this goes with the "if your problem is too big, break it up into smaller problems" approach. That paradigm stuck in my head from engineering school, and I've found it to be a very good approach to problems that are too large to solve in one pass.