[Python-Dev] Cleaning-up the new unittest API

[Michael Foord]

On 30/10/2010 06:56, Raymond Hettinger wrote:
>
> On Oct 29, 2010, at 9:11 PM, Michael Foord wrote:
>>> Just to clarify. The following fails in Python 3:
>>>
>>> sorted([3, 1, 2, None])
>>>
>>> If you want to compare that two iterables containing heterogeneous 
>>> types have the same members then it is tricky to implement correctly 
>>> and assertItemsEqual does it for you.
>>>
>>> I agree that the name is not ideal and would be happy to change the 
>>> name (deprecating the old name as it was released in 2.7). API churn 
>>> is as bad as API bloat, but at least changing the name is something 
>>> only done once.
>>
>> Sorry for the noise. Suggested alternative name:
>>
>> assertElementsEqual
>>
>> The docs need updating to make it clear that the method isn't just a 
>> synonym for assertEqual(sorted(iter1), sorted(iter2)) and that it 
>> works with unorderable types.
>
> I looked at this again and think we should just remove 
> assertItemsEqual() from Py3.2 and dedocument it in Py2.7. It is listed 
> as being new in 3.2 so nothing is lost.

As it has been released in 2.7 (and in unittest2 for earlier versions of 
Python) removing it would add another pain point for those porting from 
Python 2 to 3. From a backwards compatibility point of view this method 
has been released (it is only new in 3.2 for the Python 3 series).

Note that for this issues plus the other cleanup related topics we have 
been discussing Raymond has created issue 10273:

http://bugs.python.org/issue10273

>
> A new name like assertElementsEqual is an improvement because it 
> doesn't suggest something like assertEqual(d.items(), d.items()), but 
> it falls short in describing its key features:
>
> * the method doesn't care about order
Something that implied order would be good but we shouldn't let the 
perfect be the enemy of the good.

> * it does care about duplicates
Both the old name and the new one imply that it does care about 
duplicates (to me at least).

> * it don't need hashability
> * it can handle sequences of non-comparable types

The name doesn't imply that it needs hashability or comparable types 
either (although the latter needs to be documented as the current 
documentation could be read as saying that comparable types are needed). 
The name doesn't need to include all its *non-requirements*, it just 
needs to describe what it does.

>
> Also, I think the O(n**2) behavior is unexpected.

I agree that this should be fixed.

> There is a O(n log n) fast-path but it has a bug and needs to be 
> removed. See issue 10242.
>
Having a more efficient 'slow-path' and moving to that by default would 
fix it. The bug is only a duplicate of the bug in sorted - caused by the 
fact that sets / frozensets can't be sorted in the standard Python way 
(their less than comparison adheres to the set definition). This is 
something that will probably surprise many Python developers:

 >>> a = [{2,4}, {1,2}]
 >>> b = a[::-1]
 >>> sorted(a)
[set([2, 4]), set([1, 2])]
 >>> sorted(b)
[set([1, 2]), set([2, 4])]

(Fixing the bug in sorted would fix assertItemsEqual ;-)

As I stated in my previous email, the functionality is still useful. Add 
on the fact that this has already been released I'm -1 one removing, +1 
on fixing O(n**2) behaviour and +0 on an alternative name.

> The sole benefit over the more explicit variants like 
> assertEqual(set(a), set(b)) and assertEqual(sorted(a), sorted(b)) is 
> that it handles a somewhat rare corner case where neither of those 
> work (unordered comparison of non-compable types when you do care 
> about duplicates). That particular case doesn't come-up much and isn't 
> suggested by either the current name or its proposed replacement.
>
I have test suites littered with self.assertEqual(sorted(expected), 
sorted(actual)) - anywhere I care about the contents of a sequence but 
not about the order it is generated in (perhaps created by iteration 
over a set or dictionary). It is not uncommon for these lists to contain 
None which makes them un-sortable in Python 3. Decorating the members 
with something that allows a stable sort would fix that - and that is 
one possible fix for the efficiency issue. It would probably propagate 
the issue that sets / frozensets don't work with sorted.

> FWIW, I checked-out some other unittest suites in other languages and 
> did not find an equivalent. That strongly suggests this is YAGNI and 
> it shouldn't be added in Py3.2. There needs to be more evidence of 
> need before putting this in. And if it goes in, it needs a really good 
> name that tells what operations are hidden behind the abstraction. 
> When reading test assertion, it is vital that the reader understand 
> exactly what is being tested. It's an API fail if a reader guesses 
> that assertElementsEqual(a,b) means list(a)==list(b); the test will 
> pass unintentionally.

I agree very much that asserts need to be readable. I think 
assertSameElements is "good enough" on this score though.

All the best,

Michael

>
> See:
> http://www.phpunit.de/manual/3.4/en/api.html
> http://kentbeck.github.com/junit/javadoc/latest/
>
>
> Raymond
>
>
>
>


-- 

http://www.voidspace.org.uk/

READ CAREFULLY. By accepting and reading this email you agree,
on behalf of your employer, to release me from all obligations
and waivers arising from any and all NON-NEGOTIATED agreements,
licenses, terms-of-service, shrinkwrap, clickwrap, browsewrap,
confidentiality, non-disclosure, non-compete and acceptable use
policies (”BOGUS AGREEMENTS”) that I have entered into with your
employer, its partners, licensors, agents and assigns, in
perpetuity, without prejudice to my ongoing rights and privileges.
You further represent that you have the authority to release me
from any BOGUS AGREEMENTS on behalf of your employer.