Junit Featuring Dynamic Proxies

update: Both of these features have been rolled into a JUnit
functional testing extension called JFunc available from
sourceforge.net

I'm currently using Junit to do integration testing (or functional
testing), rather than unit testing. This involves a number of
differences from what the standard Junit methodology would suggest.
It's important to understand that the modifications I've made aren't
necessarily applicable to everyone who's doing unit testing, but for
those of you out there who feel that sometimes Junit gets in the way
of doing other sorts of testing this may help.

For instance, a lot of the information my tests require isn't
available until the test is run or is generated during runtime and
needs to be passed between the tests. Fixtures are not really
applicable to this sort of testing. Another thing that began to
happen was I started mangling my own test method names in a sort of ad
hoc way of passing arguments to the tests. To try and fix these two
situations, I've made the following extensions to Junit.

One test object for a series of tests (rather than the standard
one test instance per test)

Call test methods with arguments

For me this makes the feel of Junit much more natural. It also
provides compile time checks that wouldn't normally happen until
runtime. This is made possible by Proxies for
classes.

These two pieces of code are effectively the same (neglecting the
commented out testlet that accepts arguments), except one is cleaner,
easier to read in my opinion, and has more checks that happen at
compile time. Also the new style allows for one to optionally choose
between one test instance per test (default) or one test instance for
all tests of that type.

Download

The source is available here. It includes a
shell script to make and run the stuff. (Would somebody from the
Windows world send me a batch file.)

I will be posting a link to this site in response to some messages
being passed around in Junit mailing list. I appreciate any direct
feedback, but posting to the Junit mailing list would probably be a
better place if you're interested in a lively discussion.

update: Both of these features have been rolled into a JUnit
functional testing extension called JFunc available from
sourceforge.net

old update: In response to a request for the option to change
the assert behavior, between the standard "fatal" assertions, and the
new non-standard "non-fatal" assertions, I created a new patch. Fatal
assertions imply only one failure per test, whereas the non-fatal
assertions allow for multiple failures per test. This behavior can be
manipulated via an accessor on the Assert class (see example below).
By default it is initialized to the standard fatal assertions.

There has been a lot of discussion on the JUnit mailing list about
allowing multiple failures in a test. Some people insist that with
the right amount of refactoring there should be no need to introduce
this flexibility. I, however, found this wasn't the case (see above
as my testing might significantly differ from yours). This
restriction hampered my testing, as I had to reflect more upon the
framework than my tests themselves. Being forced to choose between
having only one assert per test which provided more information,
rather than the more logical aggregations of asserts which provided
less information.

The change isn't radical by any means. It simply allows your tests
to have multiple failures attributed to them. Which means if you're
say comparing a collection of values, you get a report back of all the
elements that didn't match, not just the first one that caused an
AssertionFailureError to be thrown.

The Swing and AWT TestRunner should work fine as well. The only
inconsistency that this has introduced is the listing of failures may
exceed the number of failed tests displayed on the status line, which
is slightly confusing since they've typically had a one to one
correspondence until now.

Update: I left TestResult.java alone in the latest patch.
In the prior patch I augmented it slightly to report the number of
unique tests that had failed, that way the status line "Tests run: x,
Failures: y, Errors: z" would still retain the familiar x = y + z, but
decided to do away with that.