On Sun, Feb 1, 2009 at 6:14 PM, Brett Cannon <brett@python.org> wrote:
On Sun, Feb 1, 2009 at 14:40, Guido van Rossum <guido@python.org> wrote:
On Sun, Feb 1, 2009 at 2:29 PM, Christian Heimes <lists@cheimes.de> wrote:
Guilherme Polo schrieb:
So.. is there any chance we can enter in agreement what features would be useful in a test discovery that could be included with Python ? I for myself do not have fancy wishes for this one, I would be happy with something that would collect unittests, inside packages and subpackages, with some fixed patterns and let me run them with test.test_support.run_unittests, or maybe something that would collect unittests and doctests and have something like run_tests in test.test_support. But then I believe this wouldn't be good enough to substitute any of the current tools, making the addition mostly useless.
I'm +1 for a simple (!) test discovery system. I'm emphasizing on simple because there are enough frameworks for elaborate unit testing.
Such a tool should
- find all modules and packages named 'tests' for a given package name
I predict that this part is where you'll have a hard time getting consensus. There are lots of different naming conventions. It would be nice if people could use the new discovery feature without having to move all their tests around.
Guido is right; this cannot be convention-based but instead configuration-based. Let me specify in a function call what package to look in and what naming convention I used for test files.
- load all subclasses of unittest.TestCase from 'tests' module or '*/tests/test*.py' files
Once you've found the test modules, TestCase subclasses are a good place to start. Though beware -- there's a pattern that defines several alternative classes in a module, e.g. one per platform, and then defines a test suite that dynamically decides which class to use. The stdlib test suite uses this in a number of places, though I can't quite remember where.
Really? I have never come across that before in the standard library, but I am sure people do stuff like that.
I'm 100% positive I've seen it. 95% sure it was in the stdlib, but it could have been somewhere else.
There is no way any tool will work in all pre-existing situations like this where people heavily rely on a test_main function to handle decisions on what to call. But as long as it is made clear how to restrict what tests are found and it is an easy solution I am sure people will adjust.
A convention that the presence of e.g. a test_main function overrides the default in-module discovery would help too.
- support some basic filtering for test cases or test functions
Or module names.
yes please! To be able to specify only the test being worked on would be very nice indeed.
The alternative of excluding tests by pattern would also be handy. As would be pointing the tool to a specific module or class. -- --Guido van Rossum (home page: http://www.python.org/~guido/)