On Fri, Aug 6, 2010 at 2:54 PM, Victoria G. Laidler <laidler at stsci.edu> wrote:
> Hi Michael & all,
>> I desperately need a clean way to run generated tests that a) get names I can control and b) attributes that can be defined and modified in the test itself that are then attached to the test case so I can access them via my report generator plugin.
testtools + testscenarios will do what you need.
> I tried to do this with nose and it has really not worked out well - we are still using it in some places but just gave up and wrote a couple of simpler Pandokia-compliant test runners. I'd rather use a community standard so if we could get this into nose2 that would be great.
>> My immediate use case is as follows:
>> - I have a directory full of datafiles that define the inputs and expected outputs for the code under test
>> - The test itself should be python code that can:
> - import bits of test code (not just code under test) from elsewhere, including the code that does the following:
> - dynamically generate the tests based on the files that are present in the directory
the load_tests or test_suite hooks permit this step - load_tests is in
python2.7 and presumably unittest2 - you can use unittest2 with the
testtols TestCase to get both this discovery and the
TestCase.addDetails API
> - name each test based on those filenames
scenarios = [(filename1, {attribute:value, attribute2:value2}), .(..), ...]
return parameterise_tests(scenarios, basic_tests)
> - invoke and execute the code under test
The stock run step will then do that.
> - perform comparisons and save data about the comparisons in dictionaries attached to the test function
you could call addDetail(name, info) in setUp; however if you are only
interested in failed comparisons, you could use a Matcher - Matchers
support get_details in their contract from testtools 0.9.4 (I think),
and that will automatically attach back to the result object. In stock
TestResult objects this is reported as a string; using a testtools
enhanced TestResult its reported as a dict of content objects so you
can tunnel json or whatever rich data you want through it.
-Rob