Cleaning-up the new unittest API
The API for the unittest module has grown fat (the documentation is approaching 2,000 lines and 10,000 words like a small book). I think we can improve learnability by focusing on the most important parts of the API. I would like to simplify and clean-up the API for the unittest module by de-documenting assertSetEqual(), assertDictEqual(), assertListEqual, and assertTupleEqual(). All of those methods are already called automatically by assertEqual(), so those methods never need to be called directly. Or, if you need to be more explicit about the type checking for sequences, the assertSequenceEqual() method will suffice. Either way, there's no need to expose the four type specific methods. Besides de-documenting those four redundant methods, I propose that assertItemsEqual() be deprecated just like its brother assertSameElements(). I haven't found anyone who accurately guesses what those methods entail based on their method names ("items" usually implies key/value pairs elsewhere in the language; nor is it clear whether order is important, whether the elements need to be hashable or orderable or just define equality tests, nor is is clear whether duplicates cause the test to fail). Given the purpose of the unittest module, it's important that the reader have a crystal clear understanding of what a test is doing. Their attention needs to be focused on the subject of the test, not on questioning the semantics of the test method. IMO, users are far better-off sticking with assertEqual() so they can be specific about the nature of the test: # hashable elements; ignore dups assertEqual(set(a), set(b)) # orderable elements; dups matter, order doesn't assertEqual(sorted(a), sorted(b)) # eq tested elements, dups matter, order matters assertEqual(list(a), list(b)) # hashable keys, eq tested values # ignore dups, ignore order assertEqual(dict(a), dict(b)) These take just a few more characters than assertSameElements() and assertItemsEqual(), but they are far more clear about their meaning. You won't have to second guess what semantics are hidden behind the abstraction. There are a couple other problems with the new API but it is probably too late to do anything about it. * elsewhere in Python we spell comparison names with abbreviations like eq, ne, lt, le, gt, ge. In unittest, those are spelled in an awkward, not easily remembered manner: assertLessEqual(a, b), etc. Fortunately, it's clear what the mean; however, it's not easy to guess their spelling. * the names for assertRegexpMatches() and assertNotRegexpMatches are deeply misleading since they are implemented in terms of re.search(), not re.match(). Raymond P.S. I also looked ar assertDictContainsSubset(a,b). It is a bit over-specialized, but at least it is crystal clear what is does and it beats the awkward alternative using dict views: assertLessEqual(a.items(), b.items())
On 29/10/2010 23:14, Raymond Hettinger wrote:
The API for the unittest module has grown fat (the documentation is approaching 2,000 lines and 10,000 words like a small book). I think we can improve learnability by focusing on the most important parts of the API.
I would like to simplify and clean-up the API for the unittest module by de-documenting assertSetEqual(), assertDictEqual(), assertListEqual, and assertTupleEqual().
All of those methods are already called automatically by assertEqual(), so those methods never need to be called directly. Or, if you need to be more explicit about the type checking for sequences, the assertSequenceEqual() method will suffice. Either way, there's no need to expose the four type specific methods.
I'm fine with dedocumenting these. These methods should not need to be called directly.
Besides de-documenting those four redundant methods, I propose that assertItemsEqual() be deprecated just like its brother assertSameElements(). I haven't found anyone who accurately guesses what those methods entail based on their method names ("items" usually implies key/value pairs elsewhere in the language; nor is it clear whether order is important, whether the elements need to be hashable or orderable or just define equality tests, nor is is clear whether duplicates cause the test to fail).
Given the purpose of the unittest module, it's important that the reader have a crystal clear understanding of what a test is doing. Their attention needs to be focused on the subject of the test, not on questioning the semantics of the test method.
assertItemsEqual compares two iterables and tests that they have the same elements irrespective of order. A relatively straightforward definition. Hopefully the docstring and documentation make this clear. If the members are all of the same type then indeed comparing two sorted lists is only slightly more typing. If the members are of different types checking that the members are the same is a much harder problem in Python 3, and this method can be very useful. -1 for deprecating. All the best, Michael Foord
IMO, users are far better-off sticking with assertEqual() so they can be specific about the nature of the test:
# hashable elements; ignore dups assertEqual(set(a), set(b))
# orderable elements; dups matter, order doesn't assertEqual(sorted(a), sorted(b))
# eq tested elements, dups matter, order matters assertEqual(list(a), list(b))
# hashable keys, eq tested values # ignore dups, ignore order assertEqual(dict(a), dict(b))
These take just a few more characters than assertSameElements() and assertItemsEqual(), but they are far more clear about their meaning. You won't have to second guess what semantics are hidden behind the abstraction.
There are a couple other problems with the new API but it is probably too late to do anything about it.
* elsewhere in Python we spell comparison names with abbreviations like eq, ne, lt, le, gt, ge. In unittest, those are spelled in an awkward, not easily remembered manner: assertLessEqual(a, b), etc. Fortunately, it's clear what the mean; however, it's not easy to guess their spelling.
* the names for assertRegexpMatches() and assertNotRegexpMatches are deeply misleading since they are implemented in terms of re.search(), not re.match().
Raymond
P.S. I also looked ar assertDictContainsSubset(a,b). It is a bit over-specialized, but at least it is crystal clear what is does and it beats the awkward alternative using dict views:
assertLessEqual(a.items(), b.items())
_______________________________________________ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/fuzzyman%40voidspace.org.u...
On 29/10/2010 23:29, Michael Foord wrote:
[snip...]
Besides de-documenting those four redundant methods, I propose that assertItemsEqual() be deprecated just like its brother assertSameElements(). I haven't found anyone who accurately guesses what those methods entail based on their method names ("items" usually implies key/value pairs elsewhere in the language; nor is it clear whether order is important, whether the elements need to be hashable or orderable or just define equality tests, nor is is clear whether duplicates cause the test to fail).
Given the purpose of the unittest module, it's important that the reader have a crystal clear understanding of what a test is doing. Their attention needs to be focused on the subject of the test, not on questioning the semantics of the test method.
assertItemsEqual compares two iterables and tests that they have the same elements irrespective of order. A relatively straightforward definition. Hopefully the docstring and documentation make this clear.
If the members are all of the same type then indeed comparing two sorted lists is only slightly more typing. If the members are of different types checking that the members are the same is a much harder problem in Python 3, and this method can be very useful.
Just to clarify. The following fails in Python 3: sorted([3, 1, 2, None]) If you want to compare that two iterables containing heterogeneous types have the same members then it is tricky to implement correctly and assertItemsEqual does it for you. I agree that the name is not ideal and would be happy to change the name (deprecating the old name as it was released in 2.7). API churn is as bad as API bloat, but at least changing the name is something only done once. All the best, Michael
-1 for deprecating.
All the best,
Michael Foord
IMO, users are far better-off sticking with assertEqual() so they can be specific about the nature of the test:
# hashable elements; ignore dups assertEqual(set(a), set(b))
# orderable elements; dups matter, order doesn't assertEqual(sorted(a), sorted(b))
# eq tested elements, dups matter, order matters assertEqual(list(a), list(b))
# hashable keys, eq tested values # ignore dups, ignore order assertEqual(dict(a), dict(b))
These take just a few more characters than assertSameElements() and assertItemsEqual(), but they are far more clear about their meaning. You won't have to second guess what semantics are hidden behind the abstraction.
There are a couple other problems with the new API but it is probably too late to do anything about it.
* elsewhere in Python we spell comparison names with abbreviations like eq, ne, lt, le, gt, ge. In unittest, those are spelled in an awkward, not easily remembered manner: assertLessEqual(a, b), etc. Fortunately, it's clear what the mean; however, it's not easy to guess their spelling.
* the names for assertRegexpMatches() and assertNotRegexpMatches are deeply misleading since they are implemented in terms of re.search(), not re.match().
Raymond
P.S. I also looked ar assertDictContainsSubset(a,b). It is a bit over-specialized, but at least it is crystal clear what is does and it beats the awkward alternative using dict views:
assertLessEqual(a.items(), b.items())
_______________________________________________ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe:http://mail.python.org/mailman/options/python-dev/fuzzyman%40voidspace.org.u...
-- http://www.voidspace.org.uk/
_______________________________________________ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/fuzzyman%40voidspace.org.u...
On 29/10/2010 23:56, Michael Foord wrote:
On 29/10/2010 23:29, Michael Foord wrote:
[snip...]
Besides de-documenting those four redundant methods, I propose that assertItemsEqual() be deprecated just like its brother assertSameElements(). I haven't found anyone who accurately guesses what those methods entail based on their method names ("items" usually implies key/value pairs elsewhere in the language; nor is it clear whether order is important, whether the elements need to be hashable or orderable or just define equality tests, nor is is clear whether duplicates cause the test to fail).
Given the purpose of the unittest module, it's important that the reader have a crystal clear understanding of what a test is doing. Their attention needs to be focused on the subject of the test, not on questioning the semantics of the test method.
assertItemsEqual compares two iterables and tests that they have the same elements irrespective of order. A relatively straightforward definition. Hopefully the docstring and documentation make this clear.
If the members are all of the same type then indeed comparing two sorted lists is only slightly more typing. If the members are of different types checking that the members are the same is a much harder problem in Python 3, and this method can be very useful.
Just to clarify. The following fails in Python 3:
sorted([3, 1, 2, None])
If you want to compare that two iterables containing heterogeneous types have the same members then it is tricky to implement correctly and assertItemsEqual does it for you.
I agree that the name is not ideal and would be happy to change the name (deprecating the old name as it was released in 2.7). API churn is as bad as API bloat, but at least changing the name is something only done once.
Sorry for the noise. Suggested alternative name: assertElementsEqual The docs need updating to make it clear that the method isn't just a synonym for assertEqual(sorted(iter1), sorted(iter2)) and that it works with unorderable types. As for "assertLessEqual" and friends, I don't find those names intuitive. In fact whilst typing this email I initially called the method "assertLessThan". For "assertRegexpMatches" I don't find it hard to understand (in natural English it makes sense even if the standard terminology for regular expressions is different). I would have preferred "assertRegex" though. All the best, Michael
All the best,
Michael
-1 for deprecating.
All the best,
Michael Foord
IMO, users are far better-off sticking with assertEqual() so they can be specific about the nature of the test:
# hashable elements; ignore dups assertEqual(set(a), set(b))
# orderable elements; dups matter, order doesn't assertEqual(sorted(a), sorted(b))
# eq tested elements, dups matter, order matters assertEqual(list(a), list(b))
# hashable keys, eq tested values # ignore dups, ignore order assertEqual(dict(a), dict(b))
These take just a few more characters than assertSameElements() and assertItemsEqual(), but they are far more clear about their meaning. You won't have to second guess what semantics are hidden behind the abstraction.
There are a couple other problems with the new API but it is probably too late to do anything about it.
* elsewhere in Python we spell comparison names with abbreviations like eq, ne, lt, le, gt, ge. In unittest, those are spelled in an awkward, not easily remembered manner: assertLessEqual(a, b), etc. Fortunately, it's clear what the mean; however, it's not easy to guess their spelling.
* the names for assertRegexpMatches() and assertNotRegexpMatches are deeply misleading since they are implemented in terms of re.search(), not re.match().
Raymond
P.S. I also looked ar assertDictContainsSubset(a,b). It is a bit over-specialized, but at least it is crystal clear what is does and it beats the awkward alternative using dict views:
assertLessEqual(a.items(), b.items())
_______________________________________________ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe:http://mail.python.org/mailman/options/python-dev/fuzzyman%40voidspace.org.u...
-- http://www.voidspace.org.uk/
_______________________________________________ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe:http://mail.python.org/mailman/options/python-dev/fuzzyman%40voidspace.org.u...
-- http://www.voidspace.org.uk/
_______________________________________________ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/fuzzyman%40voidspace.org.u...
On Oct 29, 2010, at 9:11 PM, Michael Foord wrote:
Just to clarify. The following fails in Python 3:
sorted([3, 1, 2, None])
If you want to compare that two iterables containing heterogeneous types have the same members then it is tricky to implement correctly and assertItemsEqual does it for you.
I agree that the name is not ideal and would be happy to change the name (deprecating the old name as it was released in 2.7). API churn is as bad as API bloat, but at least changing the name is something only done once.
Sorry for the noise. Suggested alternative name:
assertElementsEqual
The docs need updating to make it clear that the method isn't just a synonym for assertEqual(sorted(iter1), sorted(iter2)) and that it works with unorderable types.
I looked at this again and think we should just remove assertItemsEqual() from Py3.2 and dedocument it in Py2.7. It is listed as being new in 3.2 so nothing is lost. A new name like assertElementsEqual is an improvement because it doesn't suggest something like assertEqual(d.items(), d.items()), but it falls short in describing its key features: * the method doesn't care about order * it does care about duplicates * it don't need hashability * it can handle sequences of non-comparable types Also, I think the O(n**2) behavior is unexpected. There is a O(n log n) fast-path but it has a bug and needs to be removed. See issue 10242. The sole benefit over the more explicit variants like assertEqual(set(a), set(b)) and assertEqual(sorted(a), sorted(b)) is that it handles a somewhat rare corner case where neither of those work (unordered comparison of non-compable types when you do care about duplicates). That particular case doesn't come-up much and isn't suggested by either the current name or its proposed replacement. FWIW, I checked-out some other unittest suites in other languages and did not find an equivalent. That strongly suggests this is YAGNI and it shouldn't be added in Py3.2. There needs to be more evidence of need before putting this in. And if it goes in, it needs a really good name that tells what operations are hidden behind the abstraction. When reading test assertion, it is vital that the reader understand exactly what is being tested. It's an API fail if a reader guesses that assertElementsEqual(a,b) means list(a)==list(b); the test will pass unintentionally. See: http://www.phpunit.de/manual/3.4/en/api.html http://kentbeck.github.com/junit/javadoc/latest/ Raymond
Raymond Hettinger <raymond.hettinger@gmail.com> wrote:
I looked at this again and think we should just remove assertItemsEqual() from Py3.2 and dedocument it in Py2.7. It is listed as being new in 3.2 so nothing is lost.
One thing that would be lost is that correct Python 2.7 code using assertItemsEqual would no longer run on 3.2.
The sole benefit over the more explicit variants like assertEqual(set(a), set(b)) and assertEqual(sorted(a), sorted(b)) is that it handles a somewhat rare corner case where neither of those work (unordered comparison of non-compable types when you do care about duplicates).
Another benefit is that it gives better descriptions of differences. See http://bugs.python.org/issue9977 for an example. -M-
On 30/10/2010 06:56, Raymond Hettinger wrote:
On Oct 29, 2010, at 9:11 PM, Michael Foord wrote:
Just to clarify. The following fails in Python 3:
sorted([3, 1, 2, None])
If you want to compare that two iterables containing heterogeneous types have the same members then it is tricky to implement correctly and assertItemsEqual does it for you.
I agree that the name is not ideal and would be happy to change the name (deprecating the old name as it was released in 2.7). API churn is as bad as API bloat, but at least changing the name is something only done once.
Sorry for the noise. Suggested alternative name:
assertElementsEqual
The docs need updating to make it clear that the method isn't just a synonym for assertEqual(sorted(iter1), sorted(iter2)) and that it works with unorderable types.
I looked at this again and think we should just remove assertItemsEqual() from Py3.2 and dedocument it in Py2.7. It is listed as being new in 3.2 so nothing is lost.
As it has been released in 2.7 (and in unittest2 for earlier versions of Python) removing it would add another pain point for those porting from Python 2 to 3. From a backwards compatibility point of view this method has been released (it is only new in 3.2 for the Python 3 series). Note that for this issues plus the other cleanup related topics we have been discussing Raymond has created issue 10273: http://bugs.python.org/issue10273
A new name like assertElementsEqual is an improvement because it doesn't suggest something like assertEqual(d.items(), d.items()), but it falls short in describing its key features:
* the method doesn't care about order
Something that implied order would be good but we shouldn't let the perfect be the enemy of the good.
* it does care about duplicates Both the old name and the new one imply that it does care about duplicates (to me at least).
* it don't need hashability * it can handle sequences of non-comparable types
The name doesn't imply that it needs hashability or comparable types either (although the latter needs to be documented as the current documentation could be read as saying that comparable types are needed). The name doesn't need to include all its *non-requirements*, it just needs to describe what it does.
Also, I think the O(n**2) behavior is unexpected.
I agree that this should be fixed.
There is a O(n log n) fast-path but it has a bug and needs to be removed. See issue 10242.
Having a more efficient 'slow-path' and moving to that by default would fix it. The bug is only a duplicate of the bug in sorted - caused by the fact that sets / frozensets can't be sorted in the standard Python way (their less than comparison adheres to the set definition). This is something that will probably surprise many Python developers:
a = [{2,4}, {1,2}] b = a[::-1] sorted(a) [set([2, 4]), set([1, 2])] sorted(b) [set([1, 2]), set([2, 4])]
(Fixing the bug in sorted would fix assertItemsEqual ;-) As I stated in my previous email, the functionality is still useful. Add on the fact that this has already been released I'm -1 one removing, +1 on fixing O(n**2) behaviour and +0 on an alternative name.
The sole benefit over the more explicit variants like assertEqual(set(a), set(b)) and assertEqual(sorted(a), sorted(b)) is that it handles a somewhat rare corner case where neither of those work (unordered comparison of non-compable types when you do care about duplicates). That particular case doesn't come-up much and isn't suggested by either the current name or its proposed replacement.
I have test suites littered with self.assertEqual(sorted(expected), sorted(actual)) - anywhere I care about the contents of a sequence but not about the order it is generated in (perhaps created by iteration over a set or dictionary). It is not uncommon for these lists to contain None which makes them un-sortable in Python 3. Decorating the members with something that allows a stable sort would fix that - and that is one possible fix for the efficiency issue. It would probably propagate the issue that sets / frozensets don't work with sorted.
FWIW, I checked-out some other unittest suites in other languages and did not find an equivalent. That strongly suggests this is YAGNI and it shouldn't be added in Py3.2. There needs to be more evidence of need before putting this in. And if it goes in, it needs a really good name that tells what operations are hidden behind the abstraction. When reading test assertion, it is vital that the reader understand exactly what is being tested. It's an API fail if a reader guesses that assertElementsEqual(a,b) means list(a)==list(b); the test will pass unintentionally.
I agree very much that asserts need to be readable. I think assertSameElements is "good enough" on this score though. All the best, Michael
See: http://www.phpunit.de/manual/3.4/en/api.html http://kentbeck.github.com/junit/javadoc/latest/
Raymond
-- http://www.voidspace.org.uk/ READ CAREFULLY. By accepting and reading this email you agree, on behalf of your employer, to release me from all obligations and waivers arising from any and all NON-NEGOTIATED agreements, licenses, terms-of-service, shrinkwrap, clickwrap, browsewrap, confidentiality, non-disclosure, non-compete and acceptable use policies (”BOGUS AGREEMENTS”) that I have entered into with your employer, its partners, licensors, agents and assigns, in perpetuity, without prejudice to my ongoing rights and privileges. You further represent that you have the authority to release me from any BOGUS AGREEMENTS on behalf of your employer.
On Mon, 01 Nov 2010 02:55:35 +0000 Michael Foord <fuzzyman@voidspace.org.uk> wrote:
Having a more efficient 'slow-path' and moving to that by default would fix it. The bug is only a duplicate of the bug in sorted - caused by the fact that sets / frozensets can't be sorted in the standard Python way (their less than comparison adheres to the set definition). This is something that will probably surprise many Python developers:
a = [{2,4}, {1,2}] b = a[::-1] sorted(a) [set([2, 4]), set([1, 2])] sorted(b) [set([1, 2]), set([2, 4])]
(Fixing the bug in sorted would fix assertItemsEqual ;-)
How is this a bug? The sort algorithm is stable, which means the above behaviour is a feature. I see no easy way of eliminating the O(n*n) issue. Custom key functions can't work in all cases. Regards Antoine.
On Mon, 01 Nov 2010 12:33:31 +0100, Antoine Pitrou <solipsis@pitrou.net> wrote:
On Mon, 01 Nov 2010 02:55:35 +0000 Michael Foord <fuzzyman@voidspace.org.uk> wrote:
Having a more efficient 'slow-path' and moving to that by default would fix it. The bug is only a duplicate of the bug in sorted - caused by the fact that sets / frozensets can't be sorted in the standard Python way (their less than comparison adheres to the set definition). This is something that will probably surprise many Python developers:
a = [{2,4}, {1,2}] b = a[::-1] sorted(a) [set([2, 4]), set([1, 2])] sorted(b) [set([1, 2]), set([2, 4])]
(Fixing the bug in sorted would fix assertItemsEqual ;-)
How is this a bug? The sort algorithm is stable, which means the above behaviour is a feature. I see no easy way of eliminating the O(n*n) issue. Custom key functions can't work in all cases.
Even granting some theoretical way to sort sets by their contents, it still wouldn't be a bug in sorted. Sorted is just using the results returned by '__lt__', which is what it should do. Special casing sets in sorted would be wrong. -- R. David Murray www.bitdance.com
On 01/11/2010 11:33, Antoine Pitrou wrote:
On Mon, 01 Nov 2010 02:55:35 +0000 Michael Foord<fuzzyman@voidspace.org.uk> wrote:
Having a more efficient 'slow-path' and moving to that by default would fix it. The bug is only a duplicate of the bug in sorted - caused by the fact that sets / frozensets can't be sorted in the standard Python way (their less than comparison adheres to the set definition). This is something that will probably surprise many Python developers:
a = [{2,4}, {1,2}] b = a[::-1] sorted(a) [set([2, 4]), set([1, 2])] sorted(b) [set([1, 2]), set([2, 4])]
(Fixing the bug in sorted would fix assertItemsEqual ;-) How is this a bug? The sort algorithm is stable, which means the above behaviour is a feature.
Well, bug is the wrong word as it is obviously an intended feature (or consequence of a feature). I still think, given the general behaviour of Python sorting, that it is unexpected. It breaks what is usually an invariant for sorting without an explicit key that sortable types sorted(l) == sorted(l[::-1]). There is however a note in the set documentation: Since sets only define partial ordering (subset relationships), the output of the list.sort() method is undefined for lists of sets.
I see no easy way of eliminating the O(n*n) issue. Custom key functions can't work in all cases.
Right. Special casing sets and frozensets would be one (particularly inelegant) way however. All the best, Michael
Regards
Antoine.
_______________________________________________ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/fuzzyman%40voidspace.org.u...
-- http://www.voidspace.org.uk/ READ CAREFULLY. By accepting and reading this email you agree, on behalf of your employer, to release me from all obligations and waivers arising from any and all NON-NEGOTIATED agreements, licenses, terms-of-service, shrinkwrap, clickwrap, browsewrap, confidentiality, non-disclosure, non-compete and acceptable use policies (”BOGUS AGREEMENTS”) that I have entered into with your employer, its partners, licensors, agents and assigns, in perpetuity, without prejudice to my ongoing rights and privileges. You further represent that you have the authority to release me from any BOGUS AGREEMENTS on behalf of your employer.
On Mon, 01 Nov 2010 14:26:19 -0000, Michael Foord <fuzzyman@voidspace.org.uk> wrote:
On 01/11/2010 11:33, Antoine Pitrou wrote:
On Mon, 01 Nov 2010 02:55:35 +0000 Michael Foord<fuzzyman@voidspace.org.uk> wrote:
Having a more efficient 'slow-path' and moving to that by default would fix it. The bug is only a duplicate of the bug in sorted - caused by the fact that sets / frozensets can't be sorted in the standard Python way (their less than comparison adheres to the set definition). This is something that will probably surprise many Python developers:
a =3D [{2,4}, {1,2}] b =3D a[::-1] sorted(a) [set([2, 4]), set([1, 2])] sorted(b) [set([1, 2]), set([2, 4])]
(Fixing the bug in sorted would fix assertItemsEqual ;-) How is this a bug? The sort algorithm is stable, which means the above behaviour is a feature.
Well, bug is the wrong word as it is obviously an intended feature (or consequence of a feature). I still think, given the general behaviour of Python sorting, that it is unexpected. It breaks what is usually an invariant for sorting without an explicit key that sortable types sorted(l) = sorted(l[::-1]).
Well, as Antoine pointed out, Python's sorting algorithm is stable, so that is in fact *not* an invariant:
x = ['abcd', 'foo'*50, 'foo'*50, 'dkke'] y = x[::-1] [id(n) for n in sorted(x)] [3073747680, 3073747904, 3073747624, 3073747512] [id(n) for n in sorted(y)] [3073747680, 3073747904, 3073747512, 3073747624]
Yes, == usually hides the fact that the *objects* are in a different order, but obviously that doesn't apply to sets :) -- R. David Murray www.bitdance.com
On 01/11/2010 15:10, R. David Murray wrote:
On Mon, 01 Nov 2010 14:26:19 -0000, Michael Foord<fuzzyman@voidspace.org.uk> wrote:
On 01/11/2010 11:33, Antoine Pitrou wrote:
On Mon, 01 Nov 2010 02:55:35 +0000 Michael Foord<fuzzyman@voidspace.org.uk> wrote:
Having a more efficient 'slow-path' and moving to that by default would fix it. The bug is only a duplicate of the bug in sorted - caused by the fact that sets / frozensets can't be sorted in the standard Python way (their less than comparison adheres to the set definition). This is something that will probably surprise many Python developers:
> a =3D [{2,4}, {1,2}] > b =3D a[::-1] > sorted(a) [set([2, 4]), set([1, 2])] > sorted(b) [set([1, 2]), set([2, 4])]
(Fixing the bug in sorted would fix assertItemsEqual ;-) How is this a bug? The sort algorithm is stable, which means the above behaviour is a feature. Well, bug is the wrong word as it is obviously an intended feature (or consequence of a feature). I still think, given the general behaviour of Python sorting, that it is unexpected. It breaks what is usually an invariant for sorting without an explicit key that sortable types sorted(l) = sorted(l[::-1]). Well, as Antoine pointed out, Python's sorting algorithm is stable, so that is in fact *not* an invariant:
x = ['abcd', 'foo'*50, 'foo'*50, 'dkke'] y = x[::-1] [id(n) for n in sorted(x)] [3073747680, 3073747904, 3073747624, 3073747512] [id(n) for n in sorted(y)] [3073747680, 3073747904, 3073747512, 3073747624]
Yes, == usually hides the fact that the *objects* are in a different order, but obviously that doesn't apply to sets :)
Sorry, that should have been sorted(l) == sorted(l[::-1]) - which *is* the case for your example above. Michael
-- R. David Murray www.bitdance.com
-- http://www.voidspace.org.uk/ READ CAREFULLY. By accepting and reading this email you agree, on behalf of your employer, to release me from all obligations and waivers arising from any and all NON-NEGOTIATED agreements, licenses, terms-of-service, shrinkwrap, clickwrap, browsewrap, confidentiality, non-disclosure, non-compete and acceptable use policies (”BOGUS AGREEMENTS”) that I have entered into with your employer, its partners, licensors, agents and assigns, in perpetuity, without prejudice to my ongoing rights and privileges. You further represent that you have the authority to release me from any BOGUS AGREEMENTS on behalf of your employer.
On Mon, 01 Nov 2010 15:14:36 -0000, Michael Foord <fuzzyman@voidspace.org.uk> wrote:
On 01/11/2010 15:10, R. David Murray wrote:
On Mon, 01 Nov 2010 14:26:19 -0000, Michael Foord<fuzzyman@voidspace.org.uk> wrote:
Well, bug is the wrong word as it is obviously an intended feature (or consequence of a feature). I still think, given the general behaviour of Python sorting, that it is unexpected. It breaks what is usually an invariant for sorting without an explicit key that sortable types sorted(l) = sorted(l[::-1]). Well, as Antoine pointed out, Python's sorting algorithm is stable, so that is in fact *not* an invariant:
x = ['abcd', 'foo'*50, 'foo'*50, 'dkke'] y = x[::-1] [id(n) for n in sorted(x)] [3073747680, 3073747904, 3073747624, 3073747512] [id(n) for n in sorted(y)] [3073747680, 3073747904, 3073747512, 3073747624]
Yes, == usually hides the fact that the *objects* are in a different order, but obviously that doesn't apply to sets :)
Sorry, that should have been sorted(l) == sorted(l[::-1]) - which *is* the case for your example above.
Yes, I know that's what you meant, that's why I said "== usually hides this". If you are restricting yourself to built in types, then your invariant is mostly true but (IMO) misleading, as set demonstrates. And it certainly doesn't have to be true for custom types, even if they don't redefine __lt__. You can argue that in a good design it should be, but as the set example indicates, there are problem domains where it is useful for it not to be. Or, to put it another way, *if* there is a bug here it would be in set, not sorted. -- R. David Murray www.bitdance.com
On Tue, Nov 2, 2010 at 1:33 AM, R. David Murray <rdmurray@bitdance.com> wrote:
Or, to put it another way, *if* there is a bug here it would be in set, not sorted.
Put me in the "it's not a bug, it's a feature" camp. Providing a "elements equal" check that doesn't rely on LT providing a total ordering is a non-trivial exercise. Looking at assertItemsEqual, I'd be inclined to insert a check that falls back to the "unorderable_list_difference" approach in the case where "expected != sorted(reversed(expected))" (only need to check the one, since if the expected values are totally ordered, while the actual values are not, this should show up when comparing the elements). It slows down the fast path a bit, but the updated function should at least handle partial orderings more correctly than it does now. Cheers, Nick. P.S. Late night post, so I may be missing something obvious... -- Nick Coghlan | ncoghlan@gmail.com | Brisbane, Australia
On 01/11/2010 16:23, Nick Coghlan wrote:
On Tue, Nov 2, 2010 at 1:33 AM, R. David Murray<rdmurray@bitdance.com> wrote:
Or, to put it another way, *if* there is a bug here it would be in set, not sorted. Put me in the "it's not a bug, it's a feature" camp. Providing a "elements equal" check that doesn't rely on LT providing a total ordering is a non-trivial exercise.
Looking at assertItemsEqual, I'd be inclined to insert a check that falls back to the "unorderable_list_difference" approach in the case where "expected != sorted(reversed(expected))"
If that is sufficient then it would be a nice way of keeping the fast path. (I'm not arguing that Antoine and R. David aren't correct in what they're saying about set ordering - I'm just saying that I was surprised and bet I'm not the only one. Bit of a dead end discussion. :-) Michael
(only need to check the one, since if the expected values are totally ordered, while the actual values are not, this should show up when comparing the elements). It slows down the fast path a bit, but the updated function should at least handle partial orderings more correctly than it does now.
Cheers, Nick.
P.S. Late night post, so I may be missing something obvious...
-- http://www.voidspace.org.uk/ READ CAREFULLY. By accepting and reading this email you agree, on behalf of your employer, to release me from all obligations and waivers arising from any and all NON-NEGOTIATED agreements, licenses, terms-of-service, shrinkwrap, clickwrap, browsewrap, confidentiality, non-disclosure, non-compete and acceptable use policies (”BOGUS AGREEMENTS”) that I have entered into with your employer, its partners, licensors, agents and assigns, in perpetuity, without prejudice to my ongoing rights and privileges. You further represent that you have the authority to release me from any BOGUS AGREEMENTS on behalf of your employer.
On Tue, Nov 2, 2010 at 2:26 AM, Michael Foord <fuzzyman@voidspace.org.uk> wrote:
On 01/11/2010 16:23, Nick Coghlan wrote:
Looking at assertItemsEqual, I'd be inclined to insert a check that falls back to the "unorderable_list_difference" approach in the case where "expected != sorted(reversed(expected))"
If that is sufficient then it would be a nice way of keeping the fast path.
As far as I can tell, that check is a valid partial ordering detector for any sequence that contains one or more pairs of items for which LT, EQ and GE are all False:
seq = [{1}, {2}] seq[0] < seq[1] False seq[0] == seq[1] False seq[0] > seq[1] False sorted(seq) [{1}, {2}] sorted(reversed(sorted(seq))) [{2}, {1}]
Obviously, if the sequence doesn't contain any such items (e.g. all subsets of each other), then it will look like a total ordering and use the fast path. I see that as an upside :) Cheers, Nick. -- Nick Coghlan | ncoghlan@gmail.com | Brisbane, Australia
On 10/31/2010 10:55 PM, Michael Foord wrote:
fact that sets / frozensets can't be sorted in the standard Python way (their less than comparison adheres to the set definition). This is something that will probably surprise many Python developers:
Any programmer who sorts (or uses functions that depend on proper sorting) should know and respect the difference between partial orders, such as set inclusion, and total orders, such as lex order of sequences. So I am surprised by the above claim ;-).
a = [{2,4}, {1,2}] b = a[::-1] sorted(a) [set([2, 4]), set([1, 2])] sorted(b) [set([1, 2]), set([2, 4])]
The bug is not in the sort method, but the attempt to sort partially ordered items, which are not properly sortable. a = [{2,4}, {1,2}] b = a[::-1] print(sorted(a,key=sorted)) #[{1, 2}, {2, 4}] print(sorted(b,key=sorted)) #[{1, 2}, {2, 4}] A test method (or internal branch) that depends on sorting to work properly could just refuse to work with sets (and frozensets). -- Terry Jan Reedy
On Mon, Nov 01, 2010 at 02:37:33PM -0400, Terry Reedy wrote:
On 10/31/2010 10:55 PM, Michael Foord wrote:
fact that sets / frozensets can't be sorted in the standard Python way (their less than comparison adheres to the set definition). This is something that will probably surprise many Python developers:
Any programmer who sorts (or uses functions that depend on proper sorting) should know and respect the difference between partial orders, such as set inclusion, and total orders, such as lex order of sequences. So I am surprised by the above claim ;-).
Huh. Count me out. I guess I don't live up to your standards. --titus p.s. Seriously? I can accept that there's a rational minimalist argument for this "feature", but arguing that it's somehow the responsibility of a programmer to *expect* this seems kind of whack. -- C. Titus Brown, ctb@msu.edu
C. Titus Brown writes:
p.s. Seriously? I can accept that there's a rational minimalist argument for this "feature",
It is a feature, even if you aren't gonna need it. I want it.<wink> Many programmers do know that sets are partially ordered by inclusion, preordered by size, and (in Python) totally ordered by memory address. There's nothing wrong with not knowing that -- these are rather abstract mathematical concepts. But it's very useful that sorted() or .sort() use <=, and it's very useful that Python so often obeys simple consistent rules, and it would be quite confusing to those who do understand that "in Python the set type is partially ordered by inclusion" if sorted() used some other relation to order collections of sets. It's not so hard to change this: class SizedSet (set): def __lt__(a, b): return length(a) < length(b) def __le__(a, b): return length(a) <= length(b) def __gt__(a, b): return length(a) > length(b) def __ge__(a, b): return length(a) >= length(b) # These two are arguable, which makes size comparison not so # great as a candidate for the OOWTDI of set.__lt__(). def __eq__(a, b): return length(a) == length(b) def __ne__(a, b): return length(a) != length(b) If there were an obvious way to compare sets for use in sorting, that way would very likely be the most useful definition for <=, too. But there isn't, really (it's pretty obvious that comparing memory addresses is implausible, but otherwise, there are lots of candidates that are at least sometimes useful). Do you think otherwise? If so, what do you propose for the OOWTDI of sorting a collection of sets?
but arguing that it's somehow the responsibility of a programmer to *expect* this seems kind of whack.
I don't quite agree that everyone should "expect exactly the implemented behavior", but I do think it's a Python *programmer's* responsibility to refrain from expecting something else in this case.
On Tue, Nov 02, 2010 at 05:28:43PM +0900, Stephen J. Turnbull wrote:
C. Titus Brown writes:
p.s. Seriously? I can accept that there's a rational minimalist argument for this "feature",
It is a feature, even if you aren't gonna need it. I want it.<wink>
Many programmers do know that sets are partially ordered by inclusion, preordered by size, and (in Python) totally ordered by memory address. There's nothing wrong with not knowing that -- these are rather abstract mathematical concepts. But it's very useful that sorted() or .sort() use <=, and it's very useful that Python so often obeys simple consistent rules, and it would be quite confusing to those who do understand that "in Python the set type is partially ordered by inclusion" if sorted() used some other relation to order collections of sets.
It's not so hard to change this:
[ ... ]
If there were an obvious way to compare sets for use in sorting, that way would very likely be the most useful definition for <=, too. But there isn't, really (it's pretty obvious that comparing memory addresses is implausible, but otherwise, there are lots of candidates that are at least sometimes useful). Do you think otherwise? If so, what do you propose for the OOWTDI of sorting a collection of sets?
I don't have one...
but arguing that it's somehow the responsibility of a programmer to *expect* this seems kind of whack.
I don't quite agree that everyone should "expect exactly the implemented behavior", but I do think it's a Python *programmer's* responsibility to refrain from expecting something else in this case.
...but, as someone who has to figure out how to teach stuff to CSE undergrads (and biology grads) I hate the statement "...any programmer should expect this..." because (unless you're going to disqualify a huge swathe of people from being programmers) it's *just not true*. I don't expect Python to cater to the lowest common denominator but we should be mindful of our audience, too. I think Python has a great advantage in not being too surprising much of the time, which helps quite a bit with learning. I hope people keep that in mind for future features. cheers, --t -- C. Titus Brown, ctb@msu.edu
On 11/2/2010 10:05 AM, C. Titus Brown wrote:
...but, as someone who has to figure out how to teach stuff to CSE undergrads (and biology grads) I hate the statement "...any programmer should expect this..."
And indeed I (intentionally) did not say that. People who are ignorant and inexperienced about something should avoid making expectations in any direction until they have read the doc and experimented a bit. What I did say in the post you responded to is "Any programmer who sorts (or uses functions that depend on proper sorting) should know and respect the difference between partial orders, such as set inclusion, and total orders, such as lex order of sequences." I should hope that you teach the difference, or rather, help students to notice what they already know. Tell them to consider that difference between sorting people by a totally ordered characteristic like height or weight and a characteristic that is at best partially ordered, like hair color or ethical character. Or have them consider the partial order dependencies between morning get-ready-for-class activities (socks before shoes versus pants and shirt in either order). They already do topological sorting every day, even if the name seems fancy. -- Terry Jan Reedy
On 11/2/2010 10:05 AM, C. Titus Brown wrote:
...but, as someone who has to figure out how to teach stuff to CSE undergrads (and biology grads) I hate the statement "...any programmer should expect this..."
And indeed I (intentionally) did not say that. People who are ignorant and inexperienced about something should avoid making expectations in any direction until they have read the doc and experimented a bit. Expectations come from consistent behaviour. sorted behaves consistently for *most* of the built-in types and will also work for custom types
On 02/11/2010 16:23, Terry Reedy wrote: that provide a 'standard' (total ordering) implementation of __lt__. It is very easy to *not realise* that a consequence of sets (and frozensets) providing partial ordering through operator overloading is that sorting is undefined for them. Particularly as it still works for other mutable collections. Worth being aware that custom implementations of standard operators will break expectations of users who aren't intimately aware of the problem domains that the specific type may be created for. All the best, Michael Foord
What I did say in the post you responded to is "Any programmer who sorts (or uses functions that depend on proper sorting) should know and respect the difference between partial orders, such as set inclusion, and total orders, such as lex order of sequences." I should hope that you teach the difference, or rather, help students to notice what they already know. Tell them to consider that difference between sorting people by a totally ordered characteristic like height or weight and a characteristic that is at best partially ordered, like hair color or ethical character. Or have them consider the partial order dependencies between morning get-ready-for-class activities (socks before shoes versus pants and shirt in either order). They already do topological sorting every day, even if the name seems fancy.
On 04:29 pm, fuzzyman@voidspace.org.uk wrote:
On 02/11/2010 16:23, Terry Reedy wrote:
On 11/2/2010 10:05 AM, C. Titus Brown wrote:
...but, as someone who has to figure out how to teach stuff to CSE undergrads (and biology grads) I hate the statement "...any programmer should expect this..."
And indeed I (intentionally) did not say that. People who are ignorant and inexperienced about something should avoid making expectations in any direction until they have read the doc and experimented a bit. Expectations come from consistent behaviour. sorted behaves consistently for *most* of the built-in types and will also work for custom types that provide a 'standard' (total ordering) implementation of __lt__.
It is very easy to *not realise* that a consequence of sets (and frozensets) providing partial ordering through operator overloading is that sorting is undefined for them.
Perhaps. The documentation for sets says this, though: Since sets only define partial ordering (subset relationships), the output of the list.sort() method is undefined for lists of sets.
Particularly as it still works for other mutable collections. Worth being aware that custom implementations of standard operators will break expectations of users who aren't intimately aware of the problem domains that the specific type may be created for.
I can't help thinking that most of this confusion is caused by using < for determining subsets. If < were not defined for sets and people had to use "set.issubset" (which exists already), then sorting a list with sets would raise an exception, a much more understandable failure mode than getting back a list in arbitrary order. Jean-Paul
On 02/11/2010 17:17, exarkun@twistedmatrix.com wrote:
On 04:29 pm, fuzzyman@voidspace.org.uk wrote:
On 02/11/2010 16:23, Terry Reedy wrote:
On 11/2/2010 10:05 AM, C. Titus Brown wrote:
...but, as someone who has to figure out how to teach stuff to CSE undergrads (and biology grads) I hate the statement "...any programmer should expect this..."
And indeed I (intentionally) did not say that. People who are ignorant and inexperienced about something should avoid making expectations in any direction until they have read the doc and experimented a bit. Expectations come from consistent behaviour. sorted behaves consistently for *most* of the built-in types and will also work for custom types that provide a 'standard' (total ordering) implementation of __lt__.
It is very easy to *not realise* that a consequence of sets (and frozensets) providing partial ordering through operator overloading is that sorting is undefined for them.
Perhaps. The documentation for sets says this, though:
Since sets only define partial ordering (subset relationships), the output of the list.sort() method is undefined for lists of sets.
Right, I did quote that exact text earlier in the thread. False expectations come when there are exceptions to otherwise-consistent behaviour.
Particularly as it still works for other mutable collections. Worth being aware that custom implementations of standard operators will break expectations of users who aren't intimately aware of the problem domains that the specific type may be created for.
I can't help thinking that most of this confusion is caused by using < for determining subsets. If < were not defined for sets and people had to use "set.issubset" (which exists already), then sorting a list with sets would raise an exception, a much more understandable failure mode than getting back a list in arbitrary order.
I agree. This is a cost of overloading operators with domain specific meanings. All the best, Michael Foord
Jean-Paul _______________________________________________ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/fuzzyman%40voidspace.org.u...
-- http://www.voidspace.org.uk/ READ CAREFULLY. By accepting and reading this email you agree, on behalf of your employer, to release me from all obligations and waivers arising from any and all NON-NEGOTIATED agreements, licenses, terms-of-service, shrinkwrap, clickwrap, browsewrap, confidentiality, non-disclosure, non-compete and acceptable use policies (”BOGUS AGREEMENTS”) that I have entered into with your employer, its partners, licensors, agents and assigns, in perpetuity, without prejudice to my ongoing rights and privileges. You further represent that you have the authority to release me from any BOGUS AGREEMENTS on behalf of your employer.
On 11/2/2010 1:23 PM, Michael Foord wrote:
Right, I did quote that exact text earlier in the thread. False expectations come when there are exceptions to otherwise-consistent behaviour.
Particularly as it still works for other mutable collections. Worth being aware that custom implementations of standard operators will break expectations of users who aren't intimately aware of the problem domains that the specific type may be created for.
I can't help thinking that most of this confusion is caused by using < for determining subsets. If < were not defined for sets and people had to use "set.issubset" (which exists already), then sorting a list with sets would raise an exception, a much more understandable failure mode than getting back a list in arbitrary order.
I agree. This is a cost of overloading operators with domain specific meanings.
I disagree. In mathematics, total ordering is a special case of partial ordering, not the other way around. Set inclusion is a standard example of non-total ordering. In everyday life, another example (other than action dependencies) are ancestry relationships. In general, acyclic directed graphs model sets with partial orders. Totally ordered linear chains, as with integers, are a special case. A Python program, for instance, is usually a non-unique topological sort of a set a statements with a non-total dependency order. This is related to a topological sort of a set of actions with a non-total dependency order. A NameError, if not due to a misspelling, is typically a result of violating one of the space or time order constraints. So I stick with my statement that a programmer should have some understanding (at least at a gut level) of non-total orders and non-unique sorts. They are a major part of what programming is. -- Terry Jan Reedy
exarkun@twistedmatrix.com wrote:
I can't help thinking that most of this confusion is caused by using < for determining subsets. If < were not defined for sets and people had to use "set.issubset" (which exists already), then sorting a list with sets would raise an exception, a much more understandable failure mode than getting back a list in arbitrary order.
Personally I think it was premature to throw out __cmp__. What should have happened instead is for __cmp__ to be augmented with a fourth outcome, "not equal but unordered". Then operations such as sorting that require a total ordering could use __cmp__ and complain if they get an unordered result. -- Greg
On Tue, Nov 2, 2010 at 11:23 AM, Terry Reedy <tjreedy@udel.edu> wrote:
What I did say in the post you responded to is "Any programmer who sorts (or uses functions that depend on proper sorting) should know and respect the difference between partial orders, such as set inclusion, and total orders, such as lex order of sequences."
FWIW (i.e. not much): before this thread if you'd asked me about partial and total orders I'd have had to run to Wikipedia real quick to figure it out. Hopefully I'm still allowed to use Python. Jacob
On Tue, Nov 2, 2010 at 12:37 PM, Jacob Kaplan-Moss <jacob@jacobian.org> wrote:
Hopefully I'm still allowed to use Python.
Definitely! Python's a great place to learn about all these things. :-) -Fred -- Fred L. Drake, Jr. <fdrake at acm.org> "A storm broke loose in my mind." --Albert Einstein
Terry Reedy writes:
ethical character. Or have them consider the partial order dependencies between morning get-ready-for-class activities (socks before shoes versus pants and shirt in either order). They already do topological sorting every day, even if the name seems fancy.
Augment the example a bit, perhaps: socks and pants before shoes, socks and pants in either order.
Raymond Hettinger wrote:
The API for the unittest module has grown fat (the documentation is approaching 2,000 lines and 10,000 words like a small book). I think we can improve learnability by focusing on the most important parts of the API.
I would like to simplify and clean-up the API for the unittest module by de-documenting assertSetEqual(), assertDictEqual(), assertListEqual, and assertTupleEqual().
While you're at it, what do you think of assertAlmostEqual? Is there a use-case for it as it exists now? As I see it, it suffers from a number of problems: (1) Comparing floats for equality to some number of decimal places is rarely (never?) the right way to do it. Comparison to some number of significant figures would be better. Specifying an absolute or relative error would be better still. (2) This leads people to write the wrong test, because assertAlmostEqual exists and is easy to use, while the right test is tricky and requires better understanding of floating point issues. Example: I recently wrote tests using assertAlmostEqual where I specified places=12, because my code should have been accurate to 12 significant figures. Worked fine for small floating point, but when I started testing values of the order of 10000, my tests started failing, because although it was correct to > 12 significant figures, five of those figures were not decimal places. (3) In the rare case that mere rounding is the right way to compare for approximate equality, assertAlmostEqual doesn't save you much: a handful of characters, that's about all. self.assertAlmostEqual(x, y, places=n) vs. self.assertEqual(round(x, n), round(y, n)) (Specifying the number of decimal places is keyword-only in 3.1. To me, that seems to be a gratuitous change from 2.x, where it could be specified as a positional argument.) -- Steven
On 30/10/2010 02:41, Steven D'Aprano wrote:
Raymond Hettinger wrote:
The API for the unittest module has grown fat (the documentation is approaching 2,000 lines and 10,000 words like a small book). I think we can improve learnability by focusing on the most important parts of the API.
I would like to simplify and clean-up the API for the unittest module by de-documenting assertSetEqual(), assertDictEqual(), assertListEqual, and assertTupleEqual().
While you're at it, what do you think of assertAlmostEqual? Is there a use-case for it as it exists now?
As I see it, it suffers from a number of problems:
(1) Comparing floats for equality to some number of decimal places is rarely (never?) the right way to do it. Comparison to some number of significant figures would be better. Specifying an absolute or relative error would be better still.
I agree, comparing floats to a specific number of decimal places has almost always been useless to me. What would be nice would be a 'delta' keyword argument allowing you to specify a difference instead. That could also allow you to compare non-numeric objects like times as well. Now where are the keys to that time machine... http://docs.python.org/dev/library/unittest.html#unittest.TestCase.assertAlm... All the best, Michael Foord
(2) This leads people to write the wrong test, because assertAlmostEqual exists and is easy to use, while the right test is tricky and requires better understanding of floating point issues.
Example: I recently wrote tests using assertAlmostEqual where I specified places=12, because my code should have been accurate to 12 significant figures. Worked fine for small floating point, but when I started testing values of the order of 10000, my tests started failing, because although it was correct to > 12 significant figures, five of those figures were not decimal places.
(3) In the rare case that mere rounding is the right way to compare for approximate equality, assertAlmostEqual doesn't save you much: a handful of characters, that's about all.
self.assertAlmostEqual(x, y, places=n)
vs.
self.assertEqual(round(x, n), round(y, n))
(Specifying the number of decimal places is keyword-only in 3.1. To me, that seems to be a gratuitous change from 2.x, where it could be specified as a positional argument.)
On Fri, 29 Oct 2010 20:14:27 -0700 Raymond Hettinger <raymond.hettinger@gmail.com> wrote:
I would like to simplify and clean-up the API for the unittest module by de-documenting assertSetEqual(), assertDictEqual(), assertListEqual, and assertTupleEqual().
+1 from me. Regards Antoine.
On Sat, 30 Oct 2010 14:51:28 +0200, Antoine Pitrou <solipsis@pitrou.net> wrote:
On Fri, 29 Oct 2010 20:14:27 -0700 Raymond Hettinger <raymond.hettinger@gmail.com> wrote:
I would like to simplify and clean-up the API for the unittest module by de-documenting assertSetEqual(), assertDictEqual(), assertListEqual, and assertTupleEqual().
+1 from me.
I don't disagree with this simplification, but given that you all want to pare down the unittest API, I'd be interested in your opinions on issue 10164. Because the assertBytesEqual method takes an optional argument, it seems like it would need to be documented, even though it would in a lot of cases just be used through assertEqual. -- R. David Murray www.bitdance.com
On Sat, 30 Oct 2010 12:02:27 -0400 "R. David Murray" <rdmurray@bitdance.com> wrote:
I don't disagree with this simplification, but given that you all want to pare down the unittest API, I'd be interested in your opinions on issue 10164. Because the assertBytesEqual method takes an optional argument, it seems like it would need to be documented, even though it would in a lot of cases just be used through assertEqual.
The optional argument doesn't look very useful. I imagine there are plenty of special cases where you could need custom splitting of bytestrings on a given byte, a regexp pattern, or along some fixed chunk length, but they are special cases. Regards Antoine.
On Sat, 30 Oct 2010 18:36:45 +0200, Antoine Pitrou <solipsis@pitrou.net> wrote:
On Sat, 30 Oct 2010 12:02:27 -0400 "R. David Murray" <rdmurray@bitdance.com> wrote:
I don't disagree with this simplification, but given that you all want to pare down the unittest API, I'd be interested in your opinions on issue 10164. Because the assertBytesEqual method takes an optional argument, it seems like it would need to be documented, even though it would in a lot of cases just be used through assertEqual.
The optional argument doesn't look very useful. I imagine there are plenty of special cases where you could need custom splitting of bytestrings on a given byte, a regexp pattern, or along some fixed chunk length, but they are special cases.
Well, I have a specific special case I need it for: comparing byte strings that are wire-format email messages. Considering how much of a pain it was to get right, I'd hate to see people have to reimplement the guts of it for each special case. Maybe a 'make_chunks' argument that takes a function that returns a list? -- R. David Murray www.bitdance.com
On Sat, 30 Oct 2010 14:24:10 -0400 "R. David Murray" <rdmurray@bitdance.com> wrote:
On Sat, 30 Oct 2010 18:36:45 +0200, Antoine Pitrou <solipsis@pitrou.net> wrote:
On Sat, 30 Oct 2010 12:02:27 -0400 "R. David Murray" <rdmurray@bitdance.com> wrote:
I don't disagree with this simplification, but given that you all want to pare down the unittest API, I'd be interested in your opinions on issue 10164. Because the assertBytesEqual method takes an optional argument, it seems like it would need to be documented, even though it would in a lot of cases just be used through assertEqual.
The optional argument doesn't look very useful. I imagine there are plenty of special cases where you could need custom splitting of bytestrings on a given byte, a regexp pattern, or along some fixed chunk length, but they are special cases.
Well, I have a specific special case I need it for: comparing byte strings that are wire-format email messages. Considering how much of a pain it was to get right, I'd hate to see people have to reimplement the guts of it for each special case. Maybe a 'make_chunks' argument that takes a function that returns a list?
Well, I was hoping that we don't need to make assertBytesEqual a public API ;) Regards Antoine.
On Oct 29, 2010, at 08:14 PM, Raymond Hettinger wrote:
I would like to simplify and clean-up the API for the unittest module by de-documenting assertSetEqual(), assertDictEqual(), assertListEqual, and assertTupleEqual().
As a general principle, I think all public API methods should be documented. That still leaves plenty of room for simplification. Some ways to address both concerns could be: - moving the documentation to an "advanced" or "complete reference" section - make the methods non-public by prepending an underscore - leaving them public but adding deprecation warnings to the code
All of those methods are already called automatically by assertEqual(), so those methods never need to be called directly. Or, if you need to be more explicit about the type checking for sequences, the assertSequenceEqual() method will suffice. Either way, there's no need to expose the four type specific methods.
It sounds like those methods should not be public then.
Given the purpose of the unittest module, it's important that the reader have a crystal clear understanding of what a test is doing. Their attention needs to be focused on the subject of the test, not on questioning the semantics of the test method.
That's different documentation than a complete reference manual. A reference manual should contain all public methods, functions, classes and attributes. It's there so that when you see some third party code that uses it, you have an authoritative description of the semantics of that method. We owe it to our users to have complete reference material. OTOH, we also owe them clear guidelines on best practices for the use of our API. Often, this is either obvious or can live in the same documentation. By sectioning the documentation, the module docs can be organized to give both a user guide with opinionated recommendations, and a complete reference manual. Cheers, -Barry
On Sun, Oct 31, 2010 at 9:11 AM, Barry Warsaw <barry@python.org> wrote:
On Oct 29, 2010, at 08:14 PM, Raymond Hettinger wrote:
I would like to simplify and clean-up the API for the unittest module by de-documenting assertSetEqual(), assertDictEqual(), assertListEqual, and assertTupleEqual().
As a general principle, I think all public API methods should be documented.
That still leaves plenty of room for simplification. Some ways to address both concerns could be:
- moving the documentation to an "advanced" or "complete reference" section
Agreed, I perfer simply deemphasizing these methods by reorganizing the documentation and mentioning in their documentation to, "just use assertEqual." De-documenting them is the first step towards causing unnecessary pain by taking either of the next two steps: - make the methods non-public by prepending an underscore
- leaving them public but adding deprecation warnings to the code
Please do not make any existing released methods from the unittest module non-public or add any deprecation warnings. That will simply cause unnecessary code churn and pain for people porting their code from one version to the next without benefiting anyone.
All of those methods are already called automatically by assertEqual(), so those methods never need to be called directly.
For new code I agree and we should document them as such. They _do_ have value on their own when called directly. They offer an explicit type check as part of their assertion and are much easier to read for that than manually passing the parameter to assertSequenceEqual(a, b, seq_type=xxx). How often they are used for that reason is hard for me to measure as the giant code base we have at work that uses these was written over the years prior to assertEqual automagically calling these methods based on input types so its uses tend to be a mix of cases where the type check doesn't matter and a small fraction of cases where it is important. The smarts about automagically calling an appropriate method from assertEqual were added during the sprints while contributing them at PyCon 2009.
Or, if you need to be more explicit about the type checking for sequences, the assertSequenceEqual() method will suffice. Either way, there's no need to expose the four type specific methods.
It sounds like those methods should not be public then.
I strongly prefer de-documenting them to making anything already released as public be nonpublic. That leads to pain for people moving to new versions of Python. There is no maintenance burden to keeping these trivial methods for the convenience of code that has already used them.
Given the purpose of the unittest module, it's important that the reader have a crystal clear understanding of what a test is doing. Their attention needs to be focused on the subject of the test, not on questioning the semantics of the test method.
That's different documentation than a complete reference manual. A reference manual should contain all public methods, functions, classes and attributes. It's there so that when you see some third party code that uses it, you have an authoritative description of the semantics of that method. We owe it to our users to have complete reference material.
OTOH, we also owe them clear guidelines on best practices for the use of our API. Often, this is either obvious or can live in the same documentation. By sectioning the documentation, the module docs can be organized to give both a user guide with opinionated recommendations, and a complete reference manual.
Agreed. -gps
On Oct 31, 2010, at 09:54 AM, Gregory P. Smith wrote:
- moving the documentation to an "advanced" or "complete reference" section
Agreed, I perfer simply deemphasizing these methods by reorganizing the documentation and mentioning in their documentation to, "just use assertEqual." De-documenting them is the first step towards causing unnecessary pain by taking either of the next two steps:
- make the methods non-public by prepending an underscore
- leaving them public but adding deprecation warnings to the code
Please do not make any existing released methods from the unittest module non-public or add any deprecation warnings. That will simply cause unnecessary code churn and pain for people porting their code from one version to the next without benefiting anyone.
I was hoping someone would get my not-too-subtle hint. :) -Barry
participants (16)
-
Antoine Pitrou -
Barry Warsaw -
C. Titus Brown -
exarkun＠twistedmatrix.com -
Fred Drake -
Greg Ewing -
Gregory P. Smith -
Jacob Kaplan-Moss -
Matthew Woodcraft -
Michael Foord -
Nick Coghlan -
R. David Murray -
Raymond Hettinger -
Stephen J. Turnbull -
Steven D'Aprano -
Terry Reedy