[Python-Dev] unittest: shortDescription, _TextTestResult and other issues
michael at voidspace.org.uk
Tue Feb 9 17:40:33 CET 2010
I've been looking at outstanding unittest issues as part of my
preparation for my PyCon talk. There are a couple of changes (minor) I'd
like to make that I thought I ought to run past Python-Dev first. If I
don't get any responses then I'll just do it, so you have been warned. :-)
The great google merge into unittest happened at PyCon last year .
This included a change to TestCase.shortDescription() so that it would
*always* include the test name, whereas previously it would return the
test docstring or None.
The problem this change solved was that tests with a docstring would not
have their name (test class and method name) reported during the test
run. Unfortunately the change broke part of twisted test running.
Reported as issue 7588:
It seems to me that the same effect (always reporting test name) can be
achieved in _TextTestResult.getDescription(). I propose to revert the
change to TestCase.shortDescription() (which has both a horrible name
and a horrible implementation and should probably be renamed
getDocstring so that what it does is obvious but never mind) and put the
change into _TextTestResult.
It annoys me that _TextTestResult is private, as you will almost
certainly want to use it or subclass it when implementing custom test
systems. I am going to rename it TextTestResult, alias the old name and
document the old name as being deprecated.
Another issue that I would like to address, but there are various
possible approaches, is issue 7559: http://bugs.python.org/issue7559
Currently loadTestsFromName catches ImportError and rethrows as
AttributeError. This is horrible (it obscures the original error) but
there are backwards compatibility issues with fixing it. There are three
1) Leave it (the default)
2) Only throw an AttributeError if the import fails due to the name
being invalid (the module not existing) otherwise allow the error
through. (A minor but less serious change in behavior).
3) A new method that turns failures into pseudo-tests that fail with the
original error when run. Possibly deprecating loadTestsFromName
I favour option 3, but can't think of a good replacement name. :-)
Despite deprecating (in the documentation - no actual deprecations
warnings I believe) a lot of the duplicate ways of doing things (assert*
favoured over fail* and assertEqual over assertEquals) we didn't include
deprecating assert_ in favour of assertTrue. I would like to add that to
the documentation. After 3.2 is out I would like to clean up the
documentation, removing mention of the deprecated methods from the
*main* documentation into a separate 'deprecated methods' section. They
currently make the documentation very untidy. The unittest page should
probably be split into several pages anyway and needs improving.
Other outstanding minor issues:
Allow dotted names for test discovery
http://bugs.python.org/issue7780 - I intend to implement this as
described in the last comment
A 'check_order' optional argument (defaulting to True) for
http://bugs.python.org/issue7832 - needs patch
The breaking of __unittest caused by splitting unittesst into a package
needs fixing. The fix needs to work when Python is run without frames
http://bugs.python.org/issue7815 - needs patch
Allow a __unittest (or similar) decorator for user implemented assert
http://bugs.python.org/issue1705520 - needs patch
Allow modules to define test_suite callable.
http://bugs.python.org/issue7501 - I propose to close as rejected. Use
Display time taken by individual tests when in verbose mode.
http://bugs.python.org/issue4080 - anyone any opinions?
Allow automatic formatting of arguments in assert* failure messages.
http://bugs.python.org/issue6966 - I propose to close as rejected
removeTest() method on TestSuite
http://bugs.python.org/issue1778410 - anyone any opinions?
expect methods (delayed fail)
http://bugs.python.org/issue3615 - any opinions? Personally I think that
the TestCase API is big enough already
All the best,
 Mostly in revision 7-837.
READ CAREFULLY. By accepting and reading this email you agree, on behalf of your employer, to release me from all obligations and waivers arising from any and all NON-NEGOTIATED agreements, licenses, terms-of-service, shrinkwrap, clickwrap, browsewrap, confidentiality, non-disclosure, non-compete and acceptable use policies (”BOGUS AGREEMENTS”) that I have entered into with your employer, its partners, licensors, agents and assigns, in perpetuity, without prejudice to my ongoing rights and privileges. You further represent that you have the authority to release me from any BOGUS AGREEMENTS on behalf of your employer.
More information about the Python-Dev