[Python-Dev] The iterator story

[Aahz]

On Sun, Jul 21, 2002, Oren Tirosh wrote:
> On Sat, Jul 20, 2002 at 10:09:23AM -0400, Aahz wrote:
>>Oren:
>>>
>>> That's the scenario that bit me too.  For me it was a little more
>>> difficult to find because it was wrapped in a few layers of chained
>>> transformations.  I can't tell by the last element in the chain
>>> whether the first one is re-iterable or not.
>>>
>>> My suggestion (which was rejected by Guido) was to raise an
>>> error when an iterator's .next() method is called afer it raises
>>> StopIteration.  This way, if I try to iterate over the result again
>>> at least I'll get and error like "IteratorExhaustedError" instead
>>> something that is indistinguishable from an iterator of an empty
>>> container. I hate silent errors.
>>
>> I'm still not understanding how this would help.  When a chainable
>> transformer gets StopIteration, it should immediately return.  What
>> else do you want to do?
>
> The tranformations are fine the way they are.  The problem is the     
> source - if the source is an exhausted iterator and you ask it for a  
> new iterator it will happily return itself and report StopIteration   
> on each .next(). This behavior is indistringuishable from a valid     
> iterator on an empty container.                                       

So the problem lies in asking the source for a new iterator, not in
trying to use it.  Making the iterator consumer responsible for handling
this seems like the wrong approach to me -- the consumer *shouldn't* be
able to tell the difference.  If you're breaking that paradigm, you
don't actually have an iterator consumer, you've got something else that
wants to use the iterator interface, *plus* some additional features.

The way Python normally handles issues like this is through
documentation.  (I.e., if your consumer requires an iterable capable of
producing multiple iterators rather than an iterator object, you document
that.)

> Right now I am using tricks like special-casing files and checking if
> iter(x) is x.  It works but I hate it.

You need to write your own wrapper or change the way your consumer works.
Special-casing files inside your consumer is a Bad Idea.
-- 
Aahz (aahz@pythoncraft.com)           <*>         http://www.pythoncraft.com/

Project Vote Smart: http://www.vote-smart.org/