Some functions that take a predicate from itertools accept None as its predicate:
list(itertools.filterfalse(None, range(10))) [0] list(itertools.groupby([0,0,1,1], None)) [(0,
), (1, )]
While others don't:
list(itertools.dropwhile(None, range(10))) Traceback (most recent call last): File "<stdin>", line 1, in <module> TypeError: 'NoneType' object is not callable
I'd be interested in writing a patch to make itertools more consistent if there's a consensus. I see two possible solutions: - Either remove None as a predicate parameter, which would not be backwards compatible. - Or make the functions that don't accept None start accepting it.
On 11/2/2016 6:03 PM, Francisco Couzo wrote:
Some functions that take a predicate from itertools accept None as its predicate:
list(itertools.filterfalse(None, range(10))) [0] list(itertools.groupby([0,0,1,1], None)) [(0,
), (1, )] While others don't:
list(itertools.dropwhile(None, range(10))) Traceback (most recent call last): File "<stdin>", line 1, in <module> TypeError: 'NoneType' object is not callable
I'd be interested in writing a patch to make itertools more consistent if there's a consensus.
I see two possible solutions:
* Either remove None as a predicate parameter, which would not be backwards compatible. * Or make the functions that don't accept None start accepting it.
I think you should post this as an enhancement proposal on the tracker. Put Raymond Hettinger, the itertools originator and maintainer, as nosy. -- Terry Jan Reedy
Francisco Couzo writes:
I'd be interested in writing a patch to make itertools more consistent if there's a consensus.
I don't understand what you mean by "consistent". I would argue that in Python, an argument of None means "use the TOOWTDI default". For "filterfalse", bool() is pretty obvious for the default predicate since most Python classes do have a boolean interpretation. For "groupby", I guess the identity is the "intuitive" default. But for dropwhile, bool() or the identity seem like obvious choices, but intuitively they're not very useful since the head of what's left is a false-y. When would you want that? Typically false-ies are "nothing to see here, people, move along" values. I guess that consideration suggest lambda x: not(x), but I don't see why one would want to spell that dropwhile(None, ...) rather than dropwhile(not, ...), especially since in many contexts what you might really want (and expect None to default to) is dropwhile(lambda x: x is None, ...). Bottom line for me is that this use (and non-use) of None is consistent with Python practice regarding defaults. So, "accepts None" vs "doesn't accept None" doesn't seem to me to be an important enough consistency to impose unobvious semantics on dropwhile(None, ...).
Stephen, dropwhile(None, ...) was an example, takewhile(None, ...) doesn't works either and it makes a lot of sense IMO. On Thu, Nov 3, 2016 at 7:07 AM, Stephen J. Turnbull < turnbull.stephen.fw@u.tsukuba.ac.jp> wrote:
Francisco Couzo writes:
I'd be interested in writing a patch to make itertools more consistent if there's a consensus.
I don't understand what you mean by "consistent". I would argue that in Python, an argument of None means "use the TOOWTDI default". For "filterfalse", bool() is pretty obvious for the default predicate since most Python classes do have a boolean interpretation. For "groupby", I guess the identity is the "intuitive" default. But for dropwhile, bool() or the identity seem like obvious choices, but intuitively they're not very useful since the head of what's left is a false-y. When would you want that? Typically false-ies are "nothing to see here, people, move along" values. I guess that consideration suggest lambda x: not(x), but I don't see why one would want to spell that dropwhile(None, ...) rather than dropwhile(not, ...), especially since in many contexts what you might really want (and expect None to default to) is dropwhile(lambda x: x is None, ...).
Bottom line for me is that this use (and non-use) of None is consistent with Python practice regarding defaults.
So, "accepts None" vs "doesn't accept None" doesn't seem to me to be an important enough consistency to impose unobvious semantics on dropwhile(None, ...).
On Wed, Nov 02, 2016 at 07:03:27PM -0300, Francisco Couzo wrote:
Some functions that take a predicate from itertools accept None as its predicate: [...] I'd be interested in writing a patch to make itertools more consistent if there's a consensus.
There's a line in PEP 8 about foolish consistency. Consistency just for the sake of consistency is not a virtue. I have no objection to adding support for None to individual itertools functions on the basis that it makes sense for that function. If you can suggest a reasonable and *useful* default predicate or key function to use for None, then sure, add it to the tracker as a feature enhancement. (And, of course, a patch would be appreciated.) But that will be because it is a useful enhancement, not because of "consistency".
I see two possible solutions:
Before there can be a solution, there first has to be a problem that needs solving. "Lack of consistency" is not necessarily a problem. The intertools functions are quite different, they do different things with different APIs. The question here should not be "why don't these functions take None as an argument?", rather it should be "why should these functions take None as an argument?". -- Steve
On Nov 3, 2016, at 3:51 PM, Steven D'Aprano
wrote: Before there can be a solution, there first has to be a problem that needs solving. "Lack of consistency" is not necessarily a problem. The intertools functions are quite different, they do different things with different APIs. The question here should not be "why don't these functions take None as an argument?", rather it should be "why should these functions take None as an argument?".
I concur with Steven who articulated the issue perfectly. There isn't an actual problem here that needs to be solved (i.e. not a single user report in 15 years indicating that the API wasn't meeting the needs for real use-cases). I'm disinclined to churn the API unless there is a real need. FWIW, groupby() has the predicate as an optional argument so that you can write groupby('aaabbc') and have it group by value (much like the key-function on sorted() is optional). The two filter variants allow None as the first argument only for historical reasons -- once "bool" came along, it would have been better to write filter(bool, someiterable) in preference to using None which is less clear about its intention. The takewhile/dropwhile tools didn't have the same constraint to match a historical API, so there was an opportunity to have a clearer API with a simpler signature. As Terry suggested, if you have other itertools feature requests, please put them on the tracker an assign them to me. Thank you, Raymond
Well sorry about it, I didn't know about that PEP 8 line, disregard this thread then :) On Fri, Nov 4, 2016 at 2:29 PM, Raymond Hettinger < raymond.hettinger@gmail.com> wrote:
On Nov 3, 2016, at 3:51 PM, Steven D'Aprano
wrote: Before there can be a solution, there first has to be a problem that needs solving. "Lack of consistency" is not necessarily a problem. The intertools functions are quite different, they do different things with different APIs. The question here should not be "why don't these functions take None as an argument?", rather it should be "why should these functions take None as an argument?".
I concur with Steven who articulated the issue perfectly. There isn't an actual problem here that needs to be solved (i.e. not a single user report in 15 years indicating that the API wasn't meeting the needs for real use-cases). I'm disinclined to churn the API unless there is a real need.
FWIW, groupby() has the predicate as an optional argument so that you can write groupby('aaabbc') and have it group by value (much like the key-function on sorted() is optional). The two filter variants allow None as the first argument only for historical reasons -- once "bool" came along, it would have been better to write filter(bool, someiterable) in preference to using None which is less clear about its intention. The takewhile/dropwhile tools didn't have the same constraint to match a historical API, so there was an opportunity to have a clearer API with a simpler signature.
As Terry suggested, if you have other itertools feature requests, please put them on the tracker an assign them to me.
Thank you,
Raymond
_______________________________________________ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/ franciscouzo%40gmail.com
participants (5)
-
Francisco Couzo
-
Raymond Hettinger
-
Stephen J. Turnbull
-
Steven D'Aprano
-
Terry Reedy