[Python-Dev] Proposed unittest changes

[Ron Adam]

Ben Finney wrote:
> Ron Adam <rrr at ronadam.com> writes:
> 
>> +1 for a simpler testing module.
> 
> I've no objection.
> 
>> Just letting you know there is interest in a lighter weight testing
>> suite.
> 
> 'doctest' is a very simple testing module, that is a very useful tool.
> 
>> Looking at the unittest discussions, it doesn't look like it is
>> getting easier to use, but more complex.
> 
> How so?
> 
> One PEP proposed this week specifies to do nothing but conform
> 'unittest' with the standard library guidelines, and remove redundant
> names. That surely makes it simpler to use.

No complaint here.  It's a good place to start.


> Another PEP specifies to add helper methods that simplify a number of
> common cases.

In my opinion adding 19 more methods makes it more complex.

I'd rather see more focus on handling test failures in general without the 
need for so many special helper functions.



> What is it you see making unittest "more complex to use"?

More methods and method signatures to learn and remember.


   assert_true(…)

   assert_false(…)

   assert_almost_equal(…)

   assert_equal(…)

   assert_not_almost_equal(…)

   assert_not_equal(…)

   assert_raises(exc_class, callable_obj, *args, **kwargs)

   assert_compare_true(op, first, second, msg=None)

   assert_in(container, member, msg=None)

   assert_is(first, second, msg=None)

   assert_less_than(first, second, msg=None)

   assert_greater_than(first, second, msg=None)

   assert_less_than_or_equal(first, second, msg=None)

   assert_greater_than_or_equal(first, second, msg=None)

   assert_members_equal(first, second, msg=None)

   assert_sequence_equal(first, second, msg=None)

   assert_raises_with_message_regex(exc_class, message_regex,
                callable_obj, *args, **kwargs)

   assert_compare_false(op, first, second, msg=None)

   assert_not_in(container, member, msg=None)

   assert_is_not(first, second, msg=None)

   assert_not_less_than(first, second, msg=None)

   assert_not_greater_than(first, second, msg=None)

   assert_not_less_than_or_equal(first, second, msg=None)

   assert_not_greater_than_or_equal(first, second, msg=None)

   assert_members_not_equal(first, second, msg=None)

   assert_sequence_not_equal(first, second, msg=None)



That comes to 26 variations of assert.



There are really only a small set of basic conditions to test for.

      correct values
      incorrect values
      unexpected exceptions
      correct exceptions
      incorrect exceptions
      missing exceptions


I think the unittest module could better handle testing of exceptions and 
distinguishing exception produced by test code from the code being tested. 
  That was painfully clear (to me) when we where fixing all the Python 3000 
tests.


>> Py.test looks very interesting, especially the test discovery parts.
>> I also agree we don't need special methods for every variation of
>> assertedness.
> 
> My main complaint about 'py.test' is that it's yet-another-framework.
> We already have 'doctest' and 'unittest', and they play together
> reasonably well.

I love doctest.  Because it is very different in how it works, I don't 
think it competes with more formal testing methods.


> 'nose' <URL:http://somethingaboutorange.com/mrl/projects/nose/> looks
> better for consideration, especially since it integrates well with
> 'unittest'.

Thanks for the link!  I'll give 'nose' a try next time I write tests.

>> I've been thinking that a few decorators may go a long way to making
>> writing tests easy. It also reduces the level of indentation needed.
> 
> There are a number of these already in 'nose'.

Yes, I looked at the decorator and think it is very nice and simple to use.


If something like "nose" was a submodule of unittest, unittest may be able 
to use some of it's parts.  Maybe gradually the more java like unittest 
classes and methods could be depreciated?


Cheers,
    Ron