__iter__ implies __contains__?
Hello, I honestly didn't know we exposed such semantics, and I'm wondering if the functionality is worth the astonishement:
Basically, io.StringIO provides iteration (it yields lines of text) and containment is apparently inferred from that. Regards Antoine.
On 2011-10-01, at 20:13 , Antoine Pitrou wrote:
The resolution is defined in the expressions doc[0]:
the latter kicks in any time an object with no __iter__ and a __getitem__ is tentatively iterated, I've made that error a few times with insufficiently defined dict-like objects finding themselves (rightly or wrongly) being iterated. I don't know if there's any way to make a class with an __iter__ or a __getitem__ respectively non-containing and non-iterable (apart from adding the method and raising an exception) [0] http://docs.python.org/reference/expressions.html#notin
On Sat, 1 Oct 2011 20:27:03 +0200 Masklinn <masklinn@masklinn.net> wrote:
Ah, thanks for the pointer. I think we should add a custom IOBase.__contains__ raising a TypeError, then. The current semantics don't match what most people would expect from a "file containment" predicate. Regards Antoine.
On 2011-10-01, at 20:33 , Antoine Pitrou wrote:
I think it'd also be nice to add them to some "chopping block" list for Python 4: I've yet to see these fallbacks result in anything but pain and suffering, and they can be genuinely destructive and cause hard-to- track bug, especially with modern Python code's tendency to use (non-restartable) iterators and generators (send a generator to a function which seems to take them, it performs some sort of containment check before processing, the containment consumes the generator and proceeds to not do any processing…)
On Sat, Oct 1, 2011 at 11:27 AM, Masklinn <masklinn@masklinn.net> wrote:
For comparison, it's interesting to note that collections.abc.{Iterable, Iterator} don't implement or require a __contains__() method.
There's a second fallback: Python will also try to iterate using __getitem__ and integer indexes __iter__ is not defined.
Again, for comparison, collections.abc.Sequence /does/ define default __contains__() and __iter__() methods, in terms of __len__() and __getitem__(). <snip>
Requiring the explicit marking of a class as a sequence by inheriting from the Sequence ABC in order to get such default behavior "for free" seems quite reasonable. And having containment defined by default on potentially-infinite iterators seems unwise. +1 on the suggested removals. Cheers, Chris
On Sat, Oct 1, 2011 at 4:24 PM, Chris Rebert <pyideas@rebertia.com> wrote:
-1 to any removals - fallback protocols are the heart of duck-typing and the sequence of checks here is simply the longstanding one of permitting containment tests on any iterable by default, and providing default iterators for sequences that don't provide their own. However, +1 for adding an IOBase __contains__ that raises TypeError. This will need to go through the DeprecationError dance, though (i.e. for 3.3, issue the warning before falling back on the current iteration semantics). The current semantics are strange, but it's well within the realm of possibility for someone to be relying on them. Regards, Nick. -- Nick Coghlan | ncoghlan@gmail.com | Brisbane, Australia
On 2011-10-02, at 03:01 , Nick Coghlan wrote:
In keeping with ducks, you were looking for one, didn't find anything which quacked or looked like a duck. The fallback protocol kicks in, you get something with feathers which you found near water and went "good enough". Now you drop it from 10000m because you're looking into the efficiency of a duck's flight starting airborne, and observe your quite dismayed penguin barreling towards the ground. A few seconds later, you find yourself not with additional experimental data but with a small indentation in the earth and a big mess all over it.
On Sun, Oct 2, 2011 at 7:21 AM, Masklinn <masklinn@masklinn.net> wrote:
I love that imagery :) However, it's the kind of situation that's part and parcel of duck typing - you try things and see if they work and the occasional penguin gets it in the neck. If that's inadequate for a given use case, you define an ABC and register only things you've already checked and found to behave correctly (although beware if you register Bird rather than FlyingBird - the penguins, emus and friends may still be in trouble at that point) Cheers, Nick. -- Nick Coghlan | ncoghlan@gmail.com | Brisbane, Australia
Chris Rebert wrote:
These changes don't sound even close to reasonable to me. It seems to me that the OP is making a distinction that doesn't exist. If you can write this: x = collection[0]; do_something_with(x) x = collection[1]; do_something_with(x) x = collection[2]; do_something_with(x) # ... etc. then you can write it in a loop by hand: i = -1 try: while True: i += 1 x = collection[i] do_something_with(x) except IndexError: pass But that's just a for-loop in disguise. The for-loop protocol goes all the way back to Python 1.5 and surely even older. You should, and can, be able to write this: for x in collection: do_something_with(x) Requiring collection to explicitly inherit from a Sequence ABC breaks duck typing and is anti-Pythonic. I can't comprehend a use-case where manually extracting collection[i] for sequential values of i should succeed, but doing it in a for-loop should fail. But if you have such a use-case, feel free to define __iter__ to raise an exception. Since iteration over elements is at the heart of containment tests, the same reasoning applies to __contains__. -- Steven
On Sat, Oct 1, 2011 at 6:14 PM, Steven D'Aprano <steve@pearwood.info> wrote: <snip>
Requiring collection to explicitly inherit from a Sequence ABC breaks duck typing and is anti-Pythonic.
Actually, my suggestion was just that Sequence is one possible (but clean) way to obtain the behavior; one could *of course* reimplement the functionality without recourse to Sequence if they desired. Cheers, Chris
On Sat, Oct 1, 2011 at 9:21 PM, Chris Rebert <pyideas@rebertia.com> wrote:
But why would that would forcing everyone implementation standard sequences to reimplement the wheel be an improvement over the status quo? It would be like removing the check for "x.__len__() == 0" from boolean conversions. Duck-typed fallback protocols mean that you get some behaviour for free without inheriting from anything in particular. If there's a protocol that's a problem in a given case, then disable it or *just don't use it* (e.g. people don't *do* containment tests on infinite iterators, or, if they do, they quickly learn that triggering infinite loops is a bad idea). Cheers, Nick. -- Nick Coghlan | ncoghlan@gmail.com | Brisbane, Australia
On Sat, 1 Oct 2011 22:30:21 -0400 Nick Coghlan <ncoghlan@gmail.com> wrote:
You don't reinvent the wheel if you accept to inherit from abc.Sequence. Similarly, if you want the IO stack to provide default implementations of some methods, you have to inherit from XXXIOBase. If you don't want to implement all 6 ordered comparison operators, you have to use the functools.total_ordering decorator (and this one has a bug). Regards Antoine.
Antoine Pitrou wrote:
You shouldn't be forced to inherit from abc.Sequence to implement the sequence protocols. The whole point of protocols is that they you don't need inheritance to make them work, let alone buy into the ABC mindset. If you have a class that you don't want to be iterable but otherwise obeys the iteration protocol, that is easy to fix: have __iter__ raise TypeError. An easy fix for unusual and trivial problem.
-- Steven
On Mon, 03 Oct 2011 01:10:33 +1100 Steven D'Aprano <steve@pearwood.info> wrote:
Again, nobody said you had to inherit from abc.Sequence. It just provides a convenience. If you prefer to implement everything by hand, then fine. You already have __reversed__(), index() and count() to write, so I'm not sure why __contains__() would be scary or annoying.
On Sun, Oct 2, 2011 at 10:36 AM, Antoine Pitrou <solipsis@pitrou.net> wrote:
The case can be made that index() and count() should be based on a len() style protocol rather than methods (for the same reasons that len() is a protocol rather than a direct method call). As for __reversed__, once again, it's optional and not needed for standard sequences that provide __len__ and an index based __getitem__:
It's a judgement call as to how complex an interface can be before we decide what should be protocol based, what should be optional ABC based and what should require a specific concrete class. Strings are really the only class still in the last category. Several parts of the interpreter used to require real dictionaries, but those have been slowly culled over the years. Files are complex enough that an ABC hierarchy makes sense, but even there, many operations are defined that will accept anything implementing "enough" of the relevant IO methods rather than *requiring* that they be explicitly registered with the ABCs. Collections, however, are fundamental enough that they should ideally be fully protocol based. The ABCs exist to formalise the APIs, but we shouldn't be taking fallbacks out of the rest of the interpreter just because we have shiny new hammer to play with (in fact, the fear of that happening was one of the major objections to the introduction of a formal notion of ABCs in the first place). Cheers, Nick. -- Nick Coghlan | ncoghlan@gmail.com | Brisbane, Australia
On 2011-10-02, at 16:10 , Steven D'Aprano wrote:
If you have a class that you don't want to be iterable but otherwise obeys the iteration protocol, that is easy to fix: have __iter__ raise TypeError. An easy fix for unusual and trivial problem.
Yes but you have to know it exists in the first place, and it is not obvious that `in` without `__contains__` will use `__iter__`, let alone that `iter()` without `__iter__` will use `__getitem__` (and I submit as my case study in not being obvious that *Antoine* was surprised by this behavior).
On 2011-10-02, at 03:14 , Steven D'Aprano wrote:
x = collection['foo']; do_something_with(x) x = collection['bar']; do_something_with(x) x = collection['baz']; do_something_with(x) you can't write either of the other two options, but since Python calls the exact same method, if you somehow do a containment check (or an iteration) of a simple k:v collection instead of getting a clear exception about a missing `__iter__` or `__contains__` you get a not-very-informative `KeyError: 0` 3 or 4 levels down the stack, and now have to hunt how in hell's name somebody managed to call `collection[0]`.
Masklinn wrote:
Iterators are best thought of as temporary, short-lived objects that you create when you need them and use them while they're fresh. Passing an iterator to something that is not explicitly documented as being designed for an iterator, as opposed to an iterable, is asking for trouble. It was probably a mistake not to make a clearer distinction between iterables and iterators back when the iterator protocol was designed, but we're stuck with it now. Note that this kind of problem is less likely to occur in Py3, because methods such as dict.keys() and dict.items() now return iterable views rather than iterators, so you can iterate over them multiple times without any trouble. I think this is also a good design pattern to follow when creating your own iteration-capable objects. In other words, don't write methods that return iterators directly; instead, return another object with an __iter__ method. -- Greg
On 10/2/2011 6:00 PM, Greg Ewing wrote:
It is extremely useful that iterators are iterables. The distinction needed between iterators and reiterable non-iterators is easy: def reiterable(iterable): return hasattr(iterable, '__iter__') and not hasattr(iterable, '__next__') In a context where one is going to iterate (and call __iter__ if present) more than once, only the second check is needed. Functions that need a reiterable can make that check at the start to avoid a possibly obscure message attendant on failure of reiteration. -- Terry Jan Reedy
On Sun, Oct 2, 2011 at 8:54 PM, Terry Reedy <tjreedy@udel.edu> wrote:
And I can assure you that this was no coincidence, mistake or accident. It was done very deliberately.
The distinction needed between iterators and reiterable non-iterators is easy:
Unfortunately most people must aren't going to learn rules like this. (Gee, even experienced Python programmers can't explain the relationship between __eq__ and __hash__ properly.) This is where ABCs would shine if they were used more pervasively -- you'd just assert that you had a Sequence (or a Collection or whatever) rather than having to make obscure hasattr checks for __dunder__ names. (And those hasattr checks aren't infallible. E.g. a class that defines __iter__ but not __next__ for its instances would incorrectly be accepted.) -- --Guido van Rossum (python.org/~guido)
Guido van Rossum wrote:
And I can assure you that this was no coincidence, mistake or accident. It was done very deliberately.
I know, and I understand why it seemed like a good idea at the time. It's just that my own experiences since then have led me to think that a different choice might have worked out better. Consuming an iterator is something you really don't want to do accidentally, just like you don't want to accidentally do anything else that changes the internal state of an object. The current design makes it all too easy to do just that. Passing non-reiterable objects around is not something that I think should be encouraged. Ideally, one would hardly ever see a bare iterator -- they should be like virtual particles, coming into existence when needed, performing their function and then disappearing before anyone notices they're there. I think Py3 is heading in the right direction with things like dict.keys() returning iterable views instead of iterators. Generally, we should strive to make reiterables easier to obtain and non-reiterables harder to obtain. Maybe when we've been doing that for long enough, we'll be in a position to make fallback to iteration work only for reiterables without breaking too much code. -- Greg
On Mon, Oct 3, 2011 at 12:22 AM, Greg Ewing <greg.ewing@canterbury.ac.nz> wrote:
Actually I think there is at least *some* trend in the opposite direction -- we now have a much more refined library (and even vocabulary) for "iterator algebra" than before iter() was introduced, and a subgroup of the community who can easily whip out clever ways to do things by combining iterators in new ways.
Maybe if we had introduced new syntax to iterate over a single-use iterable from the start (*requiring* to use one form or the other depending on whether iterating over a reiterable or not), we would live in a slightly better world now, but I don't think there will ever be a time when it's easy to introduce that distinction. -- --Guido van Rossum (python.org/~guido)
Guido van Rossum wrote:
Hmmm, not sure what to do about that. Maybe we should be thinking about a "reiterator algebra" to sit on top of the iterator algebra. For example, given two reiterables x and y, zip(x, y) would return a reiterable that, when iterated over, would extract iterators from x and y and return a corresponding iterator. -- Greg
On Mon, Oct 3, 2011 at 5:07 PM, Greg Ewing <greg.ewing@canterbury.ac.nz> wrote:
Can't be done in general, since one of the main points of iterator algebra is that it works with *infinite* iterators. Basically, my understanding is that iterators started life as a way of generalising various operations on containers. This is reflected in the "for x in y: assert x in y" symmetry currently ensured by the respective definitions of the two variants of 'in'. Once the iterator protocol existed, though, people realised it made possible certain things that containers can't do (such as operating on theoretically infinite data sets or data sets that won't fit in RAM all at once). The two domains are essentially disjoint (one assumes reiterability and the ability to load the whole data set into memory, while the latter denies both of those assumptions as invalid), but they share a protocol and syntax. Often, list(itr) is used to ensure the first set of assumptions holds true, while the latter is handled by carefully ensure to iterate only once over supplied iterators and using tools like itertools.tee() to preserve any state needed. I'm not sure it's actually feasible to separate the two domains cleanly at this late stage of the game, although the collections.Container ABC may get us started down that path if we explicitly register the various builtin containers with it (a duck-typed check for __contains__ would break for classes that explicitly raise TypeError, as is proposed for _BaseIO). Cheers, Nick. -- Nick Coghlan | ncoghlan@gmail.com | Brisbane, Australia
Nick Coghlan dixit (2011-10-03, 18:00): [...]
duck-typed check for __contains__ would break for classes that explicitly raise TypeError, as is proposed for _BaseIO).
Maybe there should be possible to explicitly disallow the 'in' test by setting __contains__ to None (similar to already settled __hash__=None for non-hashables). Cheers. *j
On Mon, Oct 3, 2011 at 10:35 PM, Raymond Hettinger <raymond.hettinger@gmail.com> wrote:
Indeed, __hash__ = None was a special case forced on us by the fact that object defines __hash__ and the Hashable ABC was implemented to use a duck typed instance check. The interpreter needed a way to disable hashing when users defined a custom __eq__ implementation without overriding __hash__ themselves. I suspect isinstance(obj, collections.Container) is already an adequate test to separate out "real" containers from mere iterators, and ferreting out subtle distinctions like that where duck typing isn't up to the task is one of the main reasons ABCs were added. (I earlier indicated I didn't think that was the case yet, but I subsequently realised that was due to my using isinstance() to check things at the interactive prompt when I should have been using issubclass()) Cheers, Nick. -- Nick Coghlan | ncoghlan@gmail.com | Brisbane, Australia
On Oct 3, 2011, at 3:22 AM, Greg Ewing wrote:
Passing non-reiterable objects around is not something that I think should be encouraged.
Really? Passing around iterators is a basic design pattern http://en.wikipedia.org/wiki/Iterator_pattern for lots of languages. You may have a personal programming style that avoids iterators, but that shouldn't be forced on the rest of the community. The only way for a broad categories of iterators to become reiterable is for their outputs to be stored in memory (thus defeating the just-in-time memory conserving property of many iterators). Raymond
Raymond Hettinger wrote:
Really? Passing around iterators is a basic design pattern http://en.wikipedia.org/wiki/Iterator_pattern for lots of languages.
I'm not suggesting that we stop using iterators altogether, only that reiterables are often preferable when there's a choice. Passing a reiterable to a piece of third-party library code is safer and more future-proof than passing an iterator, because it makes less assumptions about what will be done to it. There are certainly some objects that are inherently non-reiterable, such as file objects reading from pipes or sockets, but there are many others that *are* reiterable. Some of them, such as itertools.count(), are currently only available as iterators, but could just as easily be made available in a reiterable version. And the deiter() function posted earlier shows that it's always possible to construct a reiterable analogue of any iterator-algebra operator, provided you have reiterable base objects to work with.
I don't mean to force it, but to make it at least as easy to use a reiterable-based style as an iterator-based one wherever it's reasonably possible. Ideally, reiterables should be the most obvious (in the Dutch sense) choice, with iterators being the next-most-obvious choice for when you can't use reiterables.
The only way for a broad categories of iterators to become reiterable is for their outputs to be stored in memory
I'm not sure the category is as broad as all that. Note that it does *not* include infinite iterables whose elements are generated by an algorithm, such as itertools.count(). It doesn't even include disk files, however large they might be, since you can in principle open another stream reading from the same file (although the traditional way of manifesting files as objects doesn't make that as straightforward as it could be). -- Greg
Greg Ewing wrote:
I don't think it is up to the supplier of the data to try to guess what the code will do. After all, there is no limit to what silly things a called function *might* try. Why single out "iterate over an iterator twice" for special consideration? To put it another way, if a function is advertised as working on iterables (either implicitly or explicitly), I would have no compunction about passing a finite iterator and expecting it to work. If it fails to work, the bug is in the function, not my code for using an iterator. (Infinite iterators are a special case... I wouldn't expect to be able to use them in general, if for no other reason than most algorithms expect to terminate.) -- Steven
Since iteration over elements is at the heart of containment tests, the same reasoning applies to __contains__.
I was with you for the old-style sequence iteration API, but you've lost me here. I *can* imagine use-cases where "in" shouldn't work: pretty much any iterator. Doesn't it seem strange that `x in A` should succeed, but then `x in A` should fail? Devin On Sat, Oct 1, 2011 at 9:14 PM, Steven D'Aprano <steve@pearwood.info> wrote:
On 10/2/2011 4:25 PM, Guido van Rossum wrote:
I had the same reaction as Guido. Iteration is the *only* generic way to tell if an item is in a sequence or other collection . The direct hash access of sets and dicts is exceptional. The direct calculation for range is a different exception. For the other builtin sequences, and for typical iterators, which lack the information for an O(1) shortcut, 'in' (and .__contains__ if present) has to be iteration based. -- Terry Jan Reedy
Terry Reedy wrote:
I had the same reaction as Guido. Iteration is the *only* generic way to tell if an item is in a sequence or other collection.
I think the root cause of this problem is our rather cavalier attitude to the distinction between iterables and iterators. They're really quite different things, but we started out with the notion that "for x in stuff" should be equally applicable to both, and hence decided to give every iterator an __iter__ method that returns itself. By doing that, we made it impossible for any generic protocol function to reliably tell them apart. If I were designing the iterator protocol over again, I think I would start by recognising that starting a new iteration and continuing with an existing one are very different operations, and that you almost always intend the former rather than the latter. So I would declare that "for x in stuff" always implies a *new* iteration, and devise another syntax for continuing an existing one, such as "for x from stuff". I would define iterables and iterators as disjoint categories of object, and give __iter__ methods only to iterables, not iterators. However, at least until Py4k comes around, we're stuck with the present situation, which seems to include accepting that "x in y" will occasionally gobble an iterator that you were saving for later. -- Greg
On Mon, 2011-10-03 at 18:20 +1300, Greg Ewing wrote:
I like that. +1 for what ever future python it can be put in. Also, we currently don't have an InconclusiveException, which would mean; it may be True or False, but I can't tell, So handle this carefully. But it seems to me that nondeterministic results, make programmers uncomfortable. So I have a feeling that things like this would not be popular. Cheers, Ron
On Tue, 2011-10-04 at 10:28 +1300, Greg Ewing wrote:
In Regards to the "in" operator, when "in" can't be used on an iterator either because it's infinite, or would cause undesirable side effects. Also cases where an iterator is already partially consumed, that doesn't mean the value is not in the object being iterated, It's just no longer in the iterator.
Not really suggesting we have such an exception. For such a thing to work, it would require adding more state information to iterators. And of course there's nothing stopping anyone from writing there own class, and exception, if that type of feature is useful for them. Cheers, Ron
On Mon, Oct 03, 2011 at 06:31:08AM +1100, Steven D'Aprano wrote:
This is exactly the issue that is being discussed. In my very humble opinion classes that produce non-restartable iterators should not allow containment tests to use iterators. Oleg. -- Oleg Broytman http://phdru.name/ phd@phdru.name Programmers don't die, they just GOSUB without RETURN.
On Sun, Oct 2, 2011 at 4:23 PM, Oleg Broytman <phd@phdru.name> wrote:
And that is the part that should probably be explicitly called out as advice in PEP 8 (and perhaps in the docs themselves): iterators (as opposed to iterables) should likely override __contains__ to raise TypeError, and non-container iterables (like IO objects) should likely also be set to raise TypeError if containment tests are not well-defined for the type. Whether we adopt that advice in the standard library will need to be judged on a case by case basis, since it *would* be a breach of backwards compatibility and thus may require a DeprecationWarning period. Cheers, Nick. -- Nick Coghlan | ncoghlan@gmail.com | Brisbane, Australia
On 10/2/2011 4:23 PM, Oleg Broytman wrote:
I think this is backwards. Functions that take a generic iterable as input should call iter() on the input *once* and use to iterate just once, which means not using 'in'. Functions that need a re-iterable should check for the presence of .__next__. That will exclude file objects and other iterables. -- Terry Jan Reedy
On Oct 1, 2011, at 2:13 PM, Antoine Pitrou wrote:
I honestly didn't know we exposed such semantics, and I'm wondering if the functionality is worth the astonishement:
Since both __iter__ and __contains__ are deeply tied to "in-ness", it isn't really astonishing that they are related. For many classes, if "any(elem==obj for obj in s)" is True, then "elem in s" will also be True. Conversely, it isn't unreasonable to expect this code to succeed: for elem in s: assert elem in s The decision to make __contains__ work whenever __iter__ is defined probably goes back to Py2.2. That seems to have worked out well for most users, so I don't see a reason to change that now. Raymond
On Sun, Oct 2, 2011 at 10:05 AM, Guido van Rossum <guido@python.org> wrote:
for elem in s: assert elem in s
Correction, I read this the way Raymond meant it, not the way he wrote it, and hit Send too quickly. :-( The problem here seems to be that collections/abc.py defines Iterable to have __iter__ but not __contains__, but the Python language defines the 'in' operator as trying __contains__ first, and if that is not defined, using __iter__. This is not surprising given Python's history, but it does cause some confusion when one compares the ABCs with the actual behavior. I also think that the way the ABCs have it makes more sense -- for single-use iterables (like files) the default behavior of "in" exhausts the iterator which is costly and fairly useless. Now, should we change "in" to only look for __contains__ and not fall back on __iter__? If we were debating Python 3's feature set I would probably agree with that, as a clean break with the past and a clear future. Since we're debating Python 3.3, however, I think we should just lay it to rest and use the fallback solution proposed: define __contains__ on files to raise TypeError, and leave the rest alone. Maybe make a note for Python 4. Maybe add a recommendation to PEP 8 to always implement __contains__ if you implement __iter__. But let's not break existing code that depends on the current behavior -- we have better things to do than to break perfectly fine working code in a fit of pedantry. -- --Guido van Rossum (python.org/~guido)
On 10/2/2011 1:28 PM, Guido van Rossum wrote:
That would break legitimate code that uses 'in file'. The following works as stated: if 'START\n' in f: for line in f: <process lines after the START line> else: <there are none> There would have to be a deprecation process. But see below.
[Did you mean __next__?] It seems to me better that functions that need a re-iterable non-iterator input should check for the absence of .__next__ to exclude *all* iterables, including file objects. There is no need to complicate out nice, simple, minimal iterator protocol. if hasattr(reiterable, '__next__'): raise TypeError("non-iterator required')
-- Terry Jan Reedy
On Sun, Oct 2, 2011 at 8:45 PM, Terry Reedy <tjreedy@udel.edu> wrote:
Hm. That code sample looks rather artificial. (Though now that I have seen it I can't help thinking that it might fit the bill for somebody... :-)
No, I really meant __iter__. Because in Python 4 I would be okay with not using a loop as a fallback if __contains__ doesn't exist. So if in Py3 "x in a" works by using __iter__, you would have to keep it working in Py4 by defining __contains__. And no, I don't expect Py4 within this decade...
That's a different issue -- you're talking about preventing bad use of __iter__ in the calling class. I was talking about supporting "in" by the defining class.
Still, most people in this thread seem to agree that "x in file" works by accident, not by design, and is more likely to do harm than good, and many have in fact proposed various more serious ways of making it not work in (I presume) Py3.3. -- --Guido van Rossum (python.org/~guido)
On 2011-10-01, at 20:13 , Antoine Pitrou wrote:
The resolution is defined in the expressions doc[0]:
the latter kicks in any time an object with no __iter__ and a __getitem__ is tentatively iterated, I've made that error a few times with insufficiently defined dict-like objects finding themselves (rightly or wrongly) being iterated. I don't know if there's any way to make a class with an __iter__ or a __getitem__ respectively non-containing and non-iterable (apart from adding the method and raising an exception) [0] http://docs.python.org/reference/expressions.html#notin
On Sat, 1 Oct 2011 20:27:03 +0200 Masklinn <masklinn@masklinn.net> wrote:
Ah, thanks for the pointer. I think we should add a custom IOBase.__contains__ raising a TypeError, then. The current semantics don't match what most people would expect from a "file containment" predicate. Regards Antoine.
On 2011-10-01, at 20:33 , Antoine Pitrou wrote:
I think it'd also be nice to add them to some "chopping block" list for Python 4: I've yet to see these fallbacks result in anything but pain and suffering, and they can be genuinely destructive and cause hard-to- track bug, especially with modern Python code's tendency to use (non-restartable) iterators and generators (send a generator to a function which seems to take them, it performs some sort of containment check before processing, the containment consumes the generator and proceeds to not do any processing…)
On Sat, Oct 1, 2011 at 11:27 AM, Masklinn <masklinn@masklinn.net> wrote:
For comparison, it's interesting to note that collections.abc.{Iterable, Iterator} don't implement or require a __contains__() method.
There's a second fallback: Python will also try to iterate using __getitem__ and integer indexes __iter__ is not defined.
Again, for comparison, collections.abc.Sequence /does/ define default __contains__() and __iter__() methods, in terms of __len__() and __getitem__(). <snip>
Requiring the explicit marking of a class as a sequence by inheriting from the Sequence ABC in order to get such default behavior "for free" seems quite reasonable. And having containment defined by default on potentially-infinite iterators seems unwise. +1 on the suggested removals. Cheers, Chris
On Sat, Oct 1, 2011 at 4:24 PM, Chris Rebert <pyideas@rebertia.com> wrote:
-1 to any removals - fallback protocols are the heart of duck-typing and the sequence of checks here is simply the longstanding one of permitting containment tests on any iterable by default, and providing default iterators for sequences that don't provide their own. However, +1 for adding an IOBase __contains__ that raises TypeError. This will need to go through the DeprecationError dance, though (i.e. for 3.3, issue the warning before falling back on the current iteration semantics). The current semantics are strange, but it's well within the realm of possibility for someone to be relying on them. Regards, Nick. -- Nick Coghlan | ncoghlan@gmail.com | Brisbane, Australia
On 2011-10-02, at 03:01 , Nick Coghlan wrote:
In keeping with ducks, you were looking for one, didn't find anything which quacked or looked like a duck. The fallback protocol kicks in, you get something with feathers which you found near water and went "good enough". Now you drop it from 10000m because you're looking into the efficiency of a duck's flight starting airborne, and observe your quite dismayed penguin barreling towards the ground. A few seconds later, you find yourself not with additional experimental data but with a small indentation in the earth and a big mess all over it.
On Sun, Oct 2, 2011 at 7:21 AM, Masklinn <masklinn@masklinn.net> wrote:
I love that imagery :) However, it's the kind of situation that's part and parcel of duck typing - you try things and see if they work and the occasional penguin gets it in the neck. If that's inadequate for a given use case, you define an ABC and register only things you've already checked and found to behave correctly (although beware if you register Bird rather than FlyingBird - the penguins, emus and friends may still be in trouble at that point) Cheers, Nick. -- Nick Coghlan | ncoghlan@gmail.com | Brisbane, Australia
Chris Rebert wrote:
These changes don't sound even close to reasonable to me. It seems to me that the OP is making a distinction that doesn't exist. If you can write this: x = collection[0]; do_something_with(x) x = collection[1]; do_something_with(x) x = collection[2]; do_something_with(x) # ... etc. then you can write it in a loop by hand: i = -1 try: while True: i += 1 x = collection[i] do_something_with(x) except IndexError: pass But that's just a for-loop in disguise. The for-loop protocol goes all the way back to Python 1.5 and surely even older. You should, and can, be able to write this: for x in collection: do_something_with(x) Requiring collection to explicitly inherit from a Sequence ABC breaks duck typing and is anti-Pythonic. I can't comprehend a use-case where manually extracting collection[i] for sequential values of i should succeed, but doing it in a for-loop should fail. But if you have such a use-case, feel free to define __iter__ to raise an exception. Since iteration over elements is at the heart of containment tests, the same reasoning applies to __contains__. -- Steven
On Sat, Oct 1, 2011 at 6:14 PM, Steven D'Aprano <steve@pearwood.info> wrote: <snip>
Requiring collection to explicitly inherit from a Sequence ABC breaks duck typing and is anti-Pythonic.
Actually, my suggestion was just that Sequence is one possible (but clean) way to obtain the behavior; one could *of course* reimplement the functionality without recourse to Sequence if they desired. Cheers, Chris
On Sat, Oct 1, 2011 at 9:21 PM, Chris Rebert <pyideas@rebertia.com> wrote:
But why would that would forcing everyone implementation standard sequences to reimplement the wheel be an improvement over the status quo? It would be like removing the check for "x.__len__() == 0" from boolean conversions. Duck-typed fallback protocols mean that you get some behaviour for free without inheriting from anything in particular. If there's a protocol that's a problem in a given case, then disable it or *just don't use it* (e.g. people don't *do* containment tests on infinite iterators, or, if they do, they quickly learn that triggering infinite loops is a bad idea). Cheers, Nick. -- Nick Coghlan | ncoghlan@gmail.com | Brisbane, Australia
On Sat, 1 Oct 2011 22:30:21 -0400 Nick Coghlan <ncoghlan@gmail.com> wrote:
You don't reinvent the wheel if you accept to inherit from abc.Sequence. Similarly, if you want the IO stack to provide default implementations of some methods, you have to inherit from XXXIOBase. If you don't want to implement all 6 ordered comparison operators, you have to use the functools.total_ordering decorator (and this one has a bug). Regards Antoine.
Antoine Pitrou wrote:
You shouldn't be forced to inherit from abc.Sequence to implement the sequence protocols. The whole point of protocols is that they you don't need inheritance to make them work, let alone buy into the ABC mindset. If you have a class that you don't want to be iterable but otherwise obeys the iteration protocol, that is easy to fix: have __iter__ raise TypeError. An easy fix for unusual and trivial problem.
-- Steven
On Mon, 03 Oct 2011 01:10:33 +1100 Steven D'Aprano <steve@pearwood.info> wrote:
Again, nobody said you had to inherit from abc.Sequence. It just provides a convenience. If you prefer to implement everything by hand, then fine. You already have __reversed__(), index() and count() to write, so I'm not sure why __contains__() would be scary or annoying.
On Sun, Oct 2, 2011 at 10:36 AM, Antoine Pitrou <solipsis@pitrou.net> wrote:
The case can be made that index() and count() should be based on a len() style protocol rather than methods (for the same reasons that len() is a protocol rather than a direct method call). As for __reversed__, once again, it's optional and not needed for standard sequences that provide __len__ and an index based __getitem__:
It's a judgement call as to how complex an interface can be before we decide what should be protocol based, what should be optional ABC based and what should require a specific concrete class. Strings are really the only class still in the last category. Several parts of the interpreter used to require real dictionaries, but those have been slowly culled over the years. Files are complex enough that an ABC hierarchy makes sense, but even there, many operations are defined that will accept anything implementing "enough" of the relevant IO methods rather than *requiring* that they be explicitly registered with the ABCs. Collections, however, are fundamental enough that they should ideally be fully protocol based. The ABCs exist to formalise the APIs, but we shouldn't be taking fallbacks out of the rest of the interpreter just because we have shiny new hammer to play with (in fact, the fear of that happening was one of the major objections to the introduction of a formal notion of ABCs in the first place). Cheers, Nick. -- Nick Coghlan | ncoghlan@gmail.com | Brisbane, Australia
On 2011-10-02, at 16:10 , Steven D'Aprano wrote:
If you have a class that you don't want to be iterable but otherwise obeys the iteration protocol, that is easy to fix: have __iter__ raise TypeError. An easy fix for unusual and trivial problem.
Yes but you have to know it exists in the first place, and it is not obvious that `in` without `__contains__` will use `__iter__`, let alone that `iter()` without `__iter__` will use `__getitem__` (and I submit as my case study in not being obvious that *Antoine* was surprised by this behavior).
On 2011-10-02, at 03:14 , Steven D'Aprano wrote:
x = collection['foo']; do_something_with(x) x = collection['bar']; do_something_with(x) x = collection['baz']; do_something_with(x) you can't write either of the other two options, but since Python calls the exact same method, if you somehow do a containment check (or an iteration) of a simple k:v collection instead of getting a clear exception about a missing `__iter__` or `__contains__` you get a not-very-informative `KeyError: 0` 3 or 4 levels down the stack, and now have to hunt how in hell's name somebody managed to call `collection[0]`.
Masklinn wrote:
Iterators are best thought of as temporary, short-lived objects that you create when you need them and use them while they're fresh. Passing an iterator to something that is not explicitly documented as being designed for an iterator, as opposed to an iterable, is asking for trouble. It was probably a mistake not to make a clearer distinction between iterables and iterators back when the iterator protocol was designed, but we're stuck with it now. Note that this kind of problem is less likely to occur in Py3, because methods such as dict.keys() and dict.items() now return iterable views rather than iterators, so you can iterate over them multiple times without any trouble. I think this is also a good design pattern to follow when creating your own iteration-capable objects. In other words, don't write methods that return iterators directly; instead, return another object with an __iter__ method. -- Greg
On 10/2/2011 6:00 PM, Greg Ewing wrote:
It is extremely useful that iterators are iterables. The distinction needed between iterators and reiterable non-iterators is easy: def reiterable(iterable): return hasattr(iterable, '__iter__') and not hasattr(iterable, '__next__') In a context where one is going to iterate (and call __iter__ if present) more than once, only the second check is needed. Functions that need a reiterable can make that check at the start to avoid a possibly obscure message attendant on failure of reiteration. -- Terry Jan Reedy
On Sun, Oct 2, 2011 at 8:54 PM, Terry Reedy <tjreedy@udel.edu> wrote:
And I can assure you that this was no coincidence, mistake or accident. It was done very deliberately.
The distinction needed between iterators and reiterable non-iterators is easy:
Unfortunately most people must aren't going to learn rules like this. (Gee, even experienced Python programmers can't explain the relationship between __eq__ and __hash__ properly.) This is where ABCs would shine if they were used more pervasively -- you'd just assert that you had a Sequence (or a Collection or whatever) rather than having to make obscure hasattr checks for __dunder__ names. (And those hasattr checks aren't infallible. E.g. a class that defines __iter__ but not __next__ for its instances would incorrectly be accepted.) -- --Guido van Rossum (python.org/~guido)
Guido van Rossum wrote:
And I can assure you that this was no coincidence, mistake or accident. It was done very deliberately.
I know, and I understand why it seemed like a good idea at the time. It's just that my own experiences since then have led me to think that a different choice might have worked out better. Consuming an iterator is something you really don't want to do accidentally, just like you don't want to accidentally do anything else that changes the internal state of an object. The current design makes it all too easy to do just that. Passing non-reiterable objects around is not something that I think should be encouraged. Ideally, one would hardly ever see a bare iterator -- they should be like virtual particles, coming into existence when needed, performing their function and then disappearing before anyone notices they're there. I think Py3 is heading in the right direction with things like dict.keys() returning iterable views instead of iterators. Generally, we should strive to make reiterables easier to obtain and non-reiterables harder to obtain. Maybe when we've been doing that for long enough, we'll be in a position to make fallback to iteration work only for reiterables without breaking too much code. -- Greg
On Mon, Oct 3, 2011 at 12:22 AM, Greg Ewing <greg.ewing@canterbury.ac.nz> wrote:
Actually I think there is at least *some* trend in the opposite direction -- we now have a much more refined library (and even vocabulary) for "iterator algebra" than before iter() was introduced, and a subgroup of the community who can easily whip out clever ways to do things by combining iterators in new ways.
Maybe if we had introduced new syntax to iterate over a single-use iterable from the start (*requiring* to use one form or the other depending on whether iterating over a reiterable or not), we would live in a slightly better world now, but I don't think there will ever be a time when it's easy to introduce that distinction. -- --Guido van Rossum (python.org/~guido)
Guido van Rossum wrote:
Hmmm, not sure what to do about that. Maybe we should be thinking about a "reiterator algebra" to sit on top of the iterator algebra. For example, given two reiterables x and y, zip(x, y) would return a reiterable that, when iterated over, would extract iterators from x and y and return a corresponding iterator. -- Greg
On Mon, Oct 3, 2011 at 5:07 PM, Greg Ewing <greg.ewing@canterbury.ac.nz> wrote:
Can't be done in general, since one of the main points of iterator algebra is that it works with *infinite* iterators. Basically, my understanding is that iterators started life as a way of generalising various operations on containers. This is reflected in the "for x in y: assert x in y" symmetry currently ensured by the respective definitions of the two variants of 'in'. Once the iterator protocol existed, though, people realised it made possible certain things that containers can't do (such as operating on theoretically infinite data sets or data sets that won't fit in RAM all at once). The two domains are essentially disjoint (one assumes reiterability and the ability to load the whole data set into memory, while the latter denies both of those assumptions as invalid), but they share a protocol and syntax. Often, list(itr) is used to ensure the first set of assumptions holds true, while the latter is handled by carefully ensure to iterate only once over supplied iterators and using tools like itertools.tee() to preserve any state needed. I'm not sure it's actually feasible to separate the two domains cleanly at this late stage of the game, although the collections.Container ABC may get us started down that path if we explicitly register the various builtin containers with it (a duck-typed check for __contains__ would break for classes that explicitly raise TypeError, as is proposed for _BaseIO). Cheers, Nick. -- Nick Coghlan | ncoghlan@gmail.com | Brisbane, Australia
On 3 October 2011 23:00, Nick Coghlan <ncoghlan@gmail.com> wrote:
But an iterable doesn't have to be finite to be reiterable:
This makes me think that you could have a process of "lifting" an iterator-making function like zip back to iterable-making:
-- Arnaud
Nick Coghlan dixit (2011-10-03, 18:00): [...]
duck-typed check for __contains__ would break for classes that explicitly raise TypeError, as is proposed for _BaseIO).
Maybe there should be possible to explicitly disallow the 'in' test by setting __contains__ to None (similar to already settled __hash__=None for non-hashables). Cheers. *j
On Mon, Oct 3, 2011 at 10:35 PM, Raymond Hettinger <raymond.hettinger@gmail.com> wrote:
Indeed, __hash__ = None was a special case forced on us by the fact that object defines __hash__ and the Hashable ABC was implemented to use a duck typed instance check. The interpreter needed a way to disable hashing when users defined a custom __eq__ implementation without overriding __hash__ themselves. I suspect isinstance(obj, collections.Container) is already an adequate test to separate out "real" containers from mere iterators, and ferreting out subtle distinctions like that where duck typing isn't up to the task is one of the main reasons ABCs were added. (I earlier indicated I didn't think that was the case yet, but I subsequently realised that was due to my using isinstance() to check things at the interactive prompt when I should have been using issubclass()) Cheers, Nick. -- Nick Coghlan | ncoghlan@gmail.com | Brisbane, Australia
On Oct 3, 2011, at 3:22 AM, Greg Ewing wrote:
Passing non-reiterable objects around is not something that I think should be encouraged.
Really? Passing around iterators is a basic design pattern http://en.wikipedia.org/wiki/Iterator_pattern for lots of languages. You may have a personal programming style that avoids iterators, but that shouldn't be forced on the rest of the community. The only way for a broad categories of iterators to become reiterable is for their outputs to be stored in memory (thus defeating the just-in-time memory conserving property of many iterators). Raymond
Raymond Hettinger wrote:
Really? Passing around iterators is a basic design pattern http://en.wikipedia.org/wiki/Iterator_pattern for lots of languages.
I'm not suggesting that we stop using iterators altogether, only that reiterables are often preferable when there's a choice. Passing a reiterable to a piece of third-party library code is safer and more future-proof than passing an iterator, because it makes less assumptions about what will be done to it. There are certainly some objects that are inherently non-reiterable, such as file objects reading from pipes or sockets, but there are many others that *are* reiterable. Some of them, such as itertools.count(), are currently only available as iterators, but could just as easily be made available in a reiterable version. And the deiter() function posted earlier shows that it's always possible to construct a reiterable analogue of any iterator-algebra operator, provided you have reiterable base objects to work with.
I don't mean to force it, but to make it at least as easy to use a reiterable-based style as an iterator-based one wherever it's reasonably possible. Ideally, reiterables should be the most obvious (in the Dutch sense) choice, with iterators being the next-most-obvious choice for when you can't use reiterables.
The only way for a broad categories of iterators to become reiterable is for their outputs to be stored in memory
I'm not sure the category is as broad as all that. Note that it does *not* include infinite iterables whose elements are generated by an algorithm, such as itertools.count(). It doesn't even include disk files, however large they might be, since you can in principle open another stream reading from the same file (although the traditional way of manifesting files as objects doesn't make that as straightforward as it could be). -- Greg
Greg Ewing wrote:
I don't think it is up to the supplier of the data to try to guess what the code will do. After all, there is no limit to what silly things a called function *might* try. Why single out "iterate over an iterator twice" for special consideration? To put it another way, if a function is advertised as working on iterables (either implicitly or explicitly), I would have no compunction about passing a finite iterator and expecting it to work. If it fails to work, the bug is in the function, not my code for using an iterator. (Infinite iterators are a special case... I wouldn't expect to be able to use them in general, if for no other reason than most algorithms expect to terminate.) -- Steven
Since iteration over elements is at the heart of containment tests, the same reasoning applies to __contains__.
I was with you for the old-style sequence iteration API, but you've lost me here. I *can* imagine use-cases where "in" shouldn't work: pretty much any iterator. Doesn't it seem strange that `x in A` should succeed, but then `x in A` should fail? Devin On Sat, Oct 1, 2011 at 9:14 PM, Steven D'Aprano <steve@pearwood.info> wrote:
On 10/2/2011 4:25 PM, Guido van Rossum wrote:
I had the same reaction as Guido. Iteration is the *only* generic way to tell if an item is in a sequence or other collection . The direct hash access of sets and dicts is exceptional. The direct calculation for range is a different exception. For the other builtin sequences, and for typical iterators, which lack the information for an O(1) shortcut, 'in' (and .__contains__ if present) has to be iteration based. -- Terry Jan Reedy
Terry Reedy wrote:
I had the same reaction as Guido. Iteration is the *only* generic way to tell if an item is in a sequence or other collection.
I think the root cause of this problem is our rather cavalier attitude to the distinction between iterables and iterators. They're really quite different things, but we started out with the notion that "for x in stuff" should be equally applicable to both, and hence decided to give every iterator an __iter__ method that returns itself. By doing that, we made it impossible for any generic protocol function to reliably tell them apart. If I were designing the iterator protocol over again, I think I would start by recognising that starting a new iteration and continuing with an existing one are very different operations, and that you almost always intend the former rather than the latter. So I would declare that "for x in stuff" always implies a *new* iteration, and devise another syntax for continuing an existing one, such as "for x from stuff". I would define iterables and iterators as disjoint categories of object, and give __iter__ methods only to iterables, not iterators. However, at least until Py4k comes around, we're stuck with the present situation, which seems to include accepting that "x in y" will occasionally gobble an iterator that you were saving for later. -- Greg
On Mon, 2011-10-03 at 18:20 +1300, Greg Ewing wrote:
I like that. +1 for what ever future python it can be put in. Also, we currently don't have an InconclusiveException, which would mean; it may be True or False, but I can't tell, So handle this carefully. But it seems to me that nondeterministic results, make programmers uncomfortable. So I have a feeling that things like this would not be popular. Cheers, Ron
On Tue, 2011-10-04 at 10:28 +1300, Greg Ewing wrote:
In Regards to the "in" operator, when "in" can't be used on an iterator either because it's infinite, or would cause undesirable side effects. Also cases where an iterator is already partially consumed, that doesn't mean the value is not in the object being iterated, It's just no longer in the iterator.
Not really suggesting we have such an exception. For such a thing to work, it would require adding more state information to iterators. And of course there's nothing stopping anyone from writing there own class, and exception, if that type of feature is useful for them. Cheers, Ron
On Mon, Oct 03, 2011 at 06:31:08AM +1100, Steven D'Aprano wrote:
This is exactly the issue that is being discussed. In my very humble opinion classes that produce non-restartable iterators should not allow containment tests to use iterators. Oleg. -- Oleg Broytman http://phdru.name/ phd@phdru.name Programmers don't die, they just GOSUB without RETURN.
On Sun, Oct 2, 2011 at 4:23 PM, Oleg Broytman <phd@phdru.name> wrote:
And that is the part that should probably be explicitly called out as advice in PEP 8 (and perhaps in the docs themselves): iterators (as opposed to iterables) should likely override __contains__ to raise TypeError, and non-container iterables (like IO objects) should likely also be set to raise TypeError if containment tests are not well-defined for the type. Whether we adopt that advice in the standard library will need to be judged on a case by case basis, since it *would* be a breach of backwards compatibility and thus may require a DeprecationWarning period. Cheers, Nick. -- Nick Coghlan | ncoghlan@gmail.com | Brisbane, Australia
On 10/2/2011 4:23 PM, Oleg Broytman wrote:
I think this is backwards. Functions that take a generic iterable as input should call iter() on the input *once* and use to iterate just once, which means not using 'in'. Functions that need a re-iterable should check for the presence of .__next__. That will exclude file objects and other iterables. -- Terry Jan Reedy
On Oct 1, 2011, at 2:13 PM, Antoine Pitrou wrote:
I honestly didn't know we exposed such semantics, and I'm wondering if the functionality is worth the astonishement:
Since both __iter__ and __contains__ are deeply tied to "in-ness", it isn't really astonishing that they are related. For many classes, if "any(elem==obj for obj in s)" is True, then "elem in s" will also be True. Conversely, it isn't unreasonable to expect this code to succeed: for elem in s: assert elem in s The decision to make __contains__ work whenever __iter__ is defined probably goes back to Py2.2. That seems to have worked out well for most users, so I don't see a reason to change that now. Raymond
On Sun, Oct 2, 2011 at 10:05 AM, Guido van Rossum <guido@python.org> wrote:
for elem in s: assert elem in s
Correction, I read this the way Raymond meant it, not the way he wrote it, and hit Send too quickly. :-( The problem here seems to be that collections/abc.py defines Iterable to have __iter__ but not __contains__, but the Python language defines the 'in' operator as trying __contains__ first, and if that is not defined, using __iter__. This is not surprising given Python's history, but it does cause some confusion when one compares the ABCs with the actual behavior. I also think that the way the ABCs have it makes more sense -- for single-use iterables (like files) the default behavior of "in" exhausts the iterator which is costly and fairly useless. Now, should we change "in" to only look for __contains__ and not fall back on __iter__? If we were debating Python 3's feature set I would probably agree with that, as a clean break with the past and a clear future. Since we're debating Python 3.3, however, I think we should just lay it to rest and use the fallback solution proposed: define __contains__ on files to raise TypeError, and leave the rest alone. Maybe make a note for Python 4. Maybe add a recommendation to PEP 8 to always implement __contains__ if you implement __iter__. But let's not break existing code that depends on the current behavior -- we have better things to do than to break perfectly fine working code in a fit of pedantry. -- --Guido van Rossum (python.org/~guido)
On 10/2/2011 1:28 PM, Guido van Rossum wrote:
That would break legitimate code that uses 'in file'. The following works as stated: if 'START\n' in f: for line in f: <process lines after the START line> else: <there are none> There would have to be a deprecation process. But see below.
[Did you mean __next__?] It seems to me better that functions that need a re-iterable non-iterator input should check for the absence of .__next__ to exclude *all* iterables, including file objects. There is no need to complicate out nice, simple, minimal iterator protocol. if hasattr(reiterable, '__next__'): raise TypeError("non-iterator required')
-- Terry Jan Reedy
On Sun, Oct 2, 2011 at 8:45 PM, Terry Reedy <tjreedy@udel.edu> wrote:
Hm. That code sample looks rather artificial. (Though now that I have seen it I can't help thinking that it might fit the bill for somebody... :-)
No, I really meant __iter__. Because in Python 4 I would be okay with not using a loop as a fallback if __contains__ doesn't exist. So if in Py3 "x in a" works by using __iter__, you would have to keep it working in Py4 by defining __contains__. And no, I don't expect Py4 within this decade...
That's a different issue -- you're talking about preventing bad use of __iter__ in the calling class. I was talking about supporting "in" by the defining class.
Still, most people in this thread seem to agree that "x in file" works by accident, not by design, and is more likely to do harm than good, and many have in fact proposed various more serious ways of making it not work in (I presume) Py3.3. -- --Guido van Rossum (python.org/~guido)
participants (15)
-
Antoine Pitrou -
Arnaud Delobelle -
Chris Rebert -
Devin Jeanpierre -
Greg Ewing -
Guido van Rossum -
Jan Kaliszewski -
Masklinn -
Nick Coghlan -
Oleg Broytman -
Raymond Hettinger -
Ron Adam -
Steven D'Aprano -
Terry Reedy -
Victor Stinner