There's been some discussion about automatic test discovery lately. Here's a random (not in any way thought through) idea: add a builtin function test() that runs tests associated with a given function, class, module, or object. Example: >>> import myproject >>> test(myproject.MainClass) ... >>> test(myproject) ... By default, test(obj) could simply run all doctests in docstrings attached to obj. For modules, it could also look for unittest.TestCase instances, and perhaps do some more advanced test discovery. test() could implement some keyword options to control exactly what and what not to do. There could perhaps also be a corresponding __test__ method/function for implementing custom test runners. Fredrik
On Thu, 2009-03-19 at 10:59 +0100, Fredrik Johansson wrote:
There's been some discussion about automatic test discovery lately. Here's a random (not in any way thought through) idea: add a builtin function test() that runs tests associated with a given function, class, module, or object.
This takes out all of the [useful] configuration for output - parallel testing, distributed testing, testing from an IDE etc. I'd love to see something like bzr's load_tests module scope hook honoured by the default test loader. It makes test discovery compatible with test customisation. I'd be happy to put a patch together. -Rob
On Thu, 19 Mar 2009 08:59:08 pm Fredrik Johansson wrote:
There's been some discussion about automatic test discovery lately. Here's a random (not in any way thought through) idea: add a builtin function test() that runs tests associated with a given function, class, module, or object.
Improved testing is always welcome, but why a built-in? I know testing is important, but is it so common and important that we need it at our fingertips, so to speak, and can't even import a module first before running tests? What's the benefit to making it a built-in instead of part of a test module? -- Steven D'Aprano
On Thu, Mar 19, 2009 at 11:48 AM, Steven D'Aprano
On Thu, 19 Mar 2009 08:59:08 pm Fredrik Johansson wrote:
There's been some discussion about automatic test discovery lately. Here's a random (not in any way thought through) idea: add a builtin function test() that runs tests associated with a given function, class, module, or object.
Improved testing is always welcome, but why a built-in?
I know testing is important, but is it so common and important that we need it at our fingertips, so to speak, and can't even import a module first before running tests? What's the benefit to making it a built-in instead of part of a test module?
It would just be a convenience, and I'm just throwing the idea out. The advantage would be a uniform and very simple interface for testing any module, without having to know whether I should import doctest, unittest or something else (and having to remember the commands used by each framework). It would certainly not be a replacement for more advanced test frameworks. Fredrik
I think what you are really looking for is a standard API for finding
the tests associated with a module, given the module object (or
perhaps its full name), perhaps combined with a standard API for
running the tests found.
I don't think running tests is of such all-importance to warrant
adding a built-in function that wraps both the test finding and test
running APIs.
But whatever you do, don't call it 'test' -- that name is overloaded
too much as it is.
--Guido
On Fri, Mar 20, 2009 at 1:03 PM, Fredrik Johansson
On Thu, Mar 19, 2009 at 11:48 AM, Steven D'Aprano
wrote: On Thu, 19 Mar 2009 08:59:08 pm Fredrik Johansson wrote:
There's been some discussion about automatic test discovery lately. Here's a random (not in any way thought through) idea: add a builtin function test() that runs tests associated with a given function, class, module, or object.
Improved testing is always welcome, but why a built-in?
I know testing is important, but is it so common and important that we need it at our fingertips, so to speak, and can't even import a module first before running tests? What's the benefit to making it a built-in instead of part of a test module?
It would just be a convenience, and I'm just throwing the idea out.
The advantage would be a uniform and very simple interface for testing any module, without having to know whether I should import doctest, unittest or something else (and having to remember the commands used by each framework). It would certainly not be a replacement for more advanced test frameworks.
Fredrik _______________________________________________ Python-ideas mailing list Python-ideas@python.org http://mail.python.org/mailman/listinfo/python-ideas
-- --Guido van Rossum (home page: http://www.python.org/~guido/)
participants (4)
-
Fredrik Johansson
-
Guido van Rossum
-
Robert Collins
-
Steven D'Aprano