There's been some discussion about automatic test discovery lately. Here's a random (not in any way thought through) idea: add a builtin function test() that runs tests associated with a given function, class, module, or object. Example: >>> import myproject >>> test(myproject.MainClass) ... >>> test(myproject) ... By default, test(obj) could simply run all doctests in docstrings attached to obj. For modules, it could also look for unittest.TestCase instances, and perhaps do some more advanced test discovery. test() could implement some keyword options to control exactly what and what not to do. There could perhaps also be a corresponding __test__ method/function for implementing custom test runners. Fredrik
On Thu, 2009-03-19 at 10:59 +0100, Fredrik Johansson wrote:
This takes out all of the [useful] configuration for output - parallel testing, distributed testing, testing from an IDE etc. I'd love to see something like bzr's load_tests module scope hook honoured by the default test loader. It makes test discovery compatible with test customisation. I'd be happy to put a patch together. -Rob
On Thu, 19 Mar 2009 08:59:08 pm Fredrik Johansson wrote:
Improved testing is always welcome, but why a built-in? I know testing is important, but is it so common and important that we need it at our fingertips, so to speak, and can't even import a module first before running tests? What's the benefit to making it a built-in instead of part of a test module? -- Steven D'Aprano
On Thu, Mar 19, 2009 at 11:48 AM, Steven D'Aprano <steve@pearwood.info> wrote:
It would just be a convenience, and I'm just throwing the idea out. The advantage would be a uniform and very simple interface for testing any module, without having to know whether I should import doctest, unittest or something else (and having to remember the commands used by each framework). It would certainly not be a replacement for more advanced test frameworks. Fredrik
I think what you are really looking for is a standard API for finding the tests associated with a module, given the module object (or perhaps its full name), perhaps combined with a standard API for running the tests found. I don't think running tests is of such all-importance to warrant adding a built-in function that wraps both the test finding and test running APIs. But whatever you do, don't call it 'test' -- that name is overloaded too much as it is. --Guido On Fri, Mar 20, 2009 at 1:03 PM, Fredrik Johansson <fredrik.johansson@gmail.com> wrote:
-- --Guido van Rossum (home page: http://www.python.org/~guido/)
On Thu, 2009-03-19 at 10:59 +0100, Fredrik Johansson wrote:
This takes out all of the [useful] configuration for output - parallel testing, distributed testing, testing from an IDE etc. I'd love to see something like bzr's load_tests module scope hook honoured by the default test loader. It makes test discovery compatible with test customisation. I'd be happy to put a patch together. -Rob
On Thu, 19 Mar 2009 08:59:08 pm Fredrik Johansson wrote:
Improved testing is always welcome, but why a built-in? I know testing is important, but is it so common and important that we need it at our fingertips, so to speak, and can't even import a module first before running tests? What's the benefit to making it a built-in instead of part of a test module? -- Steven D'Aprano
On Thu, Mar 19, 2009 at 11:48 AM, Steven D'Aprano <steve@pearwood.info> wrote:
It would just be a convenience, and I'm just throwing the idea out. The advantage would be a uniform and very simple interface for testing any module, without having to know whether I should import doctest, unittest or something else (and having to remember the commands used by each framework). It would certainly not be a replacement for more advanced test frameworks. Fredrik
I think what you are really looking for is a standard API for finding the tests associated with a module, given the module object (or perhaps its full name), perhaps combined with a standard API for running the tests found. I don't think running tests is of such all-importance to warrant adding a built-in function that wraps both the test finding and test running APIs. But whatever you do, don't call it 'test' -- that name is overloaded too much as it is. --Guido On Fri, Mar 20, 2009 at 1:03 PM, Fredrik Johansson <fredrik.johansson@gmail.com> wrote:
-- --Guido van Rossum (home page: http://www.python.org/~guido/)
participants (4)
-
Fredrik Johansson -
Guido van Rossum -
Robert Collins -
Steven D'Aprano