For some strange reason, many folks seems to hate updating test plans. I don't know why this seems like so much work, given what a tiny task it is compared to the bulk of the code, but basically, here's how it works. You change this:

use Test::More tests => 13;

To this:

use Test::More tests => 'no_plan'; # tests => 13

And when you're done, you change it back and update the test count. Some people think this is too much work, so based upon some vim code chromatic had, I wrote the following plugin. This could use a lot of work, but you might want to save this as ~/.vim/plugin/ToggleTestPlan.vim.

Basically, that maps ,tp to ToggleTestPlan() and toggles your test plan back and forth. If you switch to 'no_plan', it leaves your cursor where it is. If you switch to tests => $num_tests, it puts your cursor on the right line to change the test number. I could add more, but since I'm such a vim scripting newbie, I figure others are better placed to fix other issues. For example, the stuff for saving and restoring position were originally called like this (ganked from another plugin):

Or even better, save it as ~/.vim/ftplugin/perl_toggle_test_plan.vim; using the ftplugin/ directory and putting perl_ at the start of its name will ensure it's only loaded for Perl files.

map ,tp :call ToggleTestPlan()<cr>

That would be better written as:

map <buffer> ,tp :call ToggleTestPlan()<cr>

That will make the mapping local to the current buffer, so it won't leak out into other (non-Perl) files you subsequently open in the same Vim session.

Why were they defined like that (with <SID>)? Who knows?

<SID> is the script ID, a unique ID which identifies the source file which has this code in it. By using it like this it allows other scripts also to define a SaveCursorPosition() function without their names clashing. Basically if you're ever defining a function which will only ever be invoked from the current file (or from a mapping defined in the current file) then use <SID>.

It can protect you when your tests end unexpectedly. For example, let's say you're running 30 tests. At some point, another programmer on your team writes a bad function which calls exit. Your tests might end prematurely and having a test plan catches that. Or maybe you've just updated a CPAN module which calls exit when it shouldn't or finds some other way of terminating your tests early. Again, having a test count will protect you. It's quite possible that a test can terminate early without any tests failing.

Another example is when someone does something like this (assumes Scalar::Util::looks_like_number() has been imported):

If that returns a different number of coordinates from what you expect, having a test plan will catch that. Admittedly, this should actually look something like this:

ok my @coordinates = $point->coordinates,
'coordinates() should return the coordinates';
is scalar @coordinates, 3, '... and it should return the correct numbe+r of them';
foreach my $num ( @coordinates ) {
ok looks_like_number($num), "... and each should be a number ($num+)";
}

With that, because you're explicitly counting the number of coordinates, the test plan is not as necessary. However, as with the exit example, a test plan not only helps out when the code is less than perfect, it also helps out when the tests are less than perfect. It's such a small thing to update and when it actually catches a problem, you'll be greatful.

Some people argue, "yeah, but I never write code that stupid so this doesn't apply to me!". That's fine. If updating the test plan is too much work for them, so be it. Me, I know I make mistakes and I expect others to make mistakes. If we didn't make mistakes, we wouldn't need the tests in the first place.

(Trivia quiz: if the above tests were in a CPAN module, how might those tests fail?)

dieing exits with a non-zero exit status and so will be caught by Test::Harness. But it is possible, if unlikely, for a test to exit early in such a way that without a plan Test::Harness won't notice the problem.

I always use a plan more because I want to notice when a different number of tests gets run, such as due to a bug I've introduced into the *.t file or a loop not running the same number of iterations.

Not that I get this whole "ok 6" idea for a test suite. I'd rather output results, include the correct output, and assert that the generated output matches the correct output.

Perhaps this problem could be looked at from another angle. That is, if we can make the number of test planned automatically calculated, there should be no maintenance required to update the number of planned tests in the first place. We are 'lazy' programmers after all.

Besides having to remember to keep the number of BEGIN blocks synchronised with the addition and removal of tests, if you ever do get out of sequence in anything other than the most trivial of tests files, working out which of the BEGIN blocks needs to be adjusted is a total pain in the neck.

This is especially true if someone else made the change that threw the sequencing out.

Examine what is said, not who speaks -- Silence betokens consent -- Love the truth but pardon error.

Why can't the module count the tests before running them? When running tests in other languages, like Ruby, or Java, or what have you, you never have to give a test count and it's no problem there. Seems like a good kind of laziness to me - and a way to avoid mistakes automatically.

But why is there a need to know the exact amount of tests beforehand? Someone mentioned catching dieing or exits, but that should be easy enough to detect. I suppose it's all down to what philosophy lies behind the test systems from the start: most other systems I've seen have setup and teardown, so it is easy to tell if eveything ran (died after test # 12). Note that this does not mean you have to have a teardown, that's in the base class. Also, they seem to catch any exits and other stuff, so you can get a pass, a fail or an error out of a test, and they usually keep on running the rest. Also quite easy to track.

Is it not possible to have a runner of that kind in Perl, or is it just that everyone sticks with the old stuff? It seems very un-lazy to have to run around updating that number all the time, counting properties manually and so on. I do not see the win.