unittest: Dynamically creating tests

[Jeremy Bowers]

I have some code that has 16 possible ways of executing (four binary
parameters). The parameters are such that I can relatively easily write a
test file that will generate 16 tests to test each combination, for
complete coverage of all the space. (I'm enjoying the meta-ness of this
because I may personally only use 3 or 4 of the combinations, but in
theory all 16 are potentially useful. By carefully writing my tests I can
ensure they all work with reasonable probability and much less then 16
times the effort.)

My problem is that I'm having trouble writing the tests in such a way that
they execute. Consider the following simplified code that demonstrates
what I am trying to do:

-------------------

import unittest

tests = unittest.TestSuite()

for i in range(5):
    class NextTest(unittest.TestCase):
        def testEquality(self):
            self.assert_(1 == 1)

    tests.addTest(NextTest)

if __name__ == '__main__':
    unittest.main()

-------------------

I can't seem to figure out how to get this to do what I want to do, which
is run the 5 tests in the subsequent NextTest classes that are created.

The code above would be ideal if it worked, but it only runs the last
instance of the NextTest and ignores the "tests" test suite, which is less
then ideal.

I also tried ending the file with

-------------------

if __name__ == '__main__':
    tr = unittest.TestResult()
    tests.run(tr)

-------------------

but that results in

-------------------

Traceback (most recent call last):
  File "testtest.py", line 18, in ?
    tests.run(tr)
  File "/usr/lib/python2.2/unittest.py", line 339, in run
    return self(result)
  File "/usr/lib/python2.2/unittest.py", line 345, in __call__
    test(result)
  File "/usr/lib/python2.2/unittest.py", line 155, in __init__
    testMethod = getattr(self, methodName)
TypeError: attribute name must be string

-------------------

which doesn't make much sense to me.

My requirement is that the test classes can be generated dynamically; I
really don't care how they are aggregated or how they are run, as long as
they *are* run. Does anybody know the best way to do this?


(On a side note, is there a way to programmatically add something to the
*current* module, while it's being created? I tried adding the classes w/
different names into the current module space, which would probably be a
solution to this problem, but I couldn't figure out how. I'm looking for
something like 

setattr(__module__, name, obj)

Starting the file off with "import filename", where filename was the name
of the file, seemed promising but did not actually work; it seems there's
a seperate module object created when that is hit which is later
overwritten by the real "filename" module when Python is done creating
it.)