unittest: collecting tests from many modules?

[George Sakkis]

"Jorgen Grahn" wrote:

> I have a set of tests in different modules: test_foo.py, test_bar.py and so
> on. All of these use the simplest possible internal layout: a number of
> classes containing test*() methods, and the good old lines at the end:
>
>   if __name__ == "__main__":
>       unittest.main()
>
> This is great, because each of the modules are runnable, and I can select
> classes or tests to run on the commandline if I want to.  However, running
> all the tests from e.g. a Makefile is not that fun; I don't get a single
> pass/fail summary across the modules.
>
> What's the best way of creating a test.py which
> - aggregates the tests from all the test_*.py modules?
> - doesn't require me to enumerate all the test classes in test.py
>   (forcing each module to define test_foo.theSuite or someting would
>   be OK though)
> - retains the ability to select tests and verbosity (-q, -v) from the
>   command line?
>
> Something like:
>
>   import unittest
>   import test_foo
>   import test_bar
>
>   if __name__ == "__main__":
>       unittest.main(modules = ['test_foo',
>                                'test_bar'])
>
> Seems to me this should be possible, since all the logic for doing it /is/
> there for the local module; I'd assume there would be a way to make unittest
> search additional modules for test classes.  But my head starts spinning
> when I read the source code ...

I had written a script to do something close to this; currently it
doesn't do any kind of aggregation, but it should be easy to extend it
as you like. What I don't like is the way it currently works: it
replaces sys.modules['__main__'] for each unit test and then it
execfile()s it, which seems like a hack. I didn't look into unittest's
internals in case there is a more elegant way around this; if there
isn't, a future version of unittest should address the automatic
aggregation of tests, as py.test does already.

The link to the script is http://rafb.net/paste/results/V0y16g97.html.

Hope this helps,
George