[Python-Dev] Proposed unittest changes

[Michael Foord]

Ben Finney wrote:
> Howdy Michael,
>
> I'm interested in the changes you're proposing for Python's 'unittest' 
> module. I am (like, I suspect, many Python coders) maintaining my own 
> set of extensions to the module across many projects, so I'd really 
> like to see many of the improvements you discuss actually in the 
> standard library.
>
> What assistance can I offer to help on this issue?
>
>   
I intend to start working on them in August, after I have finished my 
current writing commitments.

The full list of changes proposed (feel free to start - but ping me or 
the list) and not shot down was something like:

Documenting that the assert method names are to be preferred over the 'FailUnless' names (this stirred up some controversy this weekend so should probably not happen).


Adding the following new asserts:

    assertIn    (member, container, msg=None)
    assertNotIn     (member, container, msg=None)
    assertIs     (first, second, msg=None)
    assertNotIs   (first, second, msg=None)
    assertRaisesWithMessage    (exc_class, message, callable, *args, 
**keywargs)


A simple 'RunTests' function that takes a collection of modules, test 
suites or test cases and runs them with TextTestRunner - passing on 
keyword arguments to the test runner. This makes running a test suite 
easier - once you have collected all your test cases together you can 
just pass them to this function so long as you are happy with the 
default runner (possibly allowing an alternative runner class to be 
provided as a keyword argument).


Make the error messages for "assertEquals" and 
"assertNotEquals" more useful - showing the objects that compare 
incorrectly even if an explicit message is passed in. Additionally, when 
comparing lists and tuples that are the same length show the members 
(and indices?) that were different.

Possibly even providing a diff in the case of comparing strings (we have an implementation of this at work).

    assertLessThan
    assertGreaterThan
    assertLessThanOrEquals
    assertGreaterThanOrEquals

Guido suggested various variants on assertEquals:

    assertListEqual(self, list1, list2, msg=None):
    assertDictEqual(self, d1, d2, msg=None):
    assertMultiLineEqual(self, first, second, msg=Non

In my opinion these can all be provided by improving the messages from assertEquals and don't require new methods.

    assertSameElements(self, expected_seq, actual_seq, msg=None):

I usually achieve this with:

    assertEquals(set(actual), set(expected))

A convenience method might be nice, but I'm worried about the API growing in an unbounded way.


Other suggestions that weren't controversial but I might not get to:

    assertRaisesWithMessage taking a regex to match the error message

    expect methods that collect failures and report at the end of the test (allowing an individual test method to raise several errors without stopping)

    assertIsInstance and assertIsSubclass



Michael Foord

-- 

http://www.ironpythoninaction.com/
http://www.voidspace.org.uk/
http://www.trypython.org/
http://www.ironpython.info/
http://www.theotherdelia.co.uk/
http://www.resolverhacks.net/