[Python-Dev] The iterator story

[Oren Tirosh]

On Sun, Jul 21, 2002 at 10:41:08AM -0400, Aahz wrote:
> > The tranformations are fine the way they are.  The problem is the     
> > source - if the source is an exhausted iterator and you ask it for a  
> > new iterator it will happily return itself and report StopIteration   
> > on each .next(). This behavior is indistringuishable from a valid     
> > iterator on an empty container.                                       
> 
> So the problem lies in asking the source for a new iterator, not in
> trying to use it.  Making the iterator consumer responsible for handling
> this seems like the wrong approach to me -- the consumer *shouldn't* be
> able to tell the difference.  If you're breaking that paradigm, you
> don't actually have an iterator consumer, you've got something else that
> wants to use the iterator interface, *plus* some additional features.

Tuples are very much like lists except that they cannot be modified. A lot
of code that was written with lists in mind can actually use tuples. If you
pass a tuple to a function that tries to use the "additional feature" of
mutability you will get an exception.

Pipes are very much likes files except that they cannot be seeked. A lot of
code that was written with files in mind can actually use pipes. If you pass
a pipe to a function that tries to use the "additional feature" of
seekbility you will get an exception.

Iterators are very much like iterable containers except that they can only
be iterated once.  A lot of code that was written with containers in mind
can actually use iterators.  If you pass an iterator to a function that
tries to use the "additional feature" of re-iterability you...  will not get 
an exception.  You'll get nonsense results because on the second pass the 
iterator will fail silently and suddenly pretend to be an empty container.

Would you say that any code that expects a seekable file or a mutable
sequence is "breaking the paradigm"?  Why should code that expects a
re-iterable container be different from code that uses any other protocol
that has several variations and subsets/supersets?

> The way Python normally handles issues like this is through
> documentation.  (I.e., if your consumer requires an iterable capable of
> producing multiple iterators rather than an iterator object, you document
> that.)

The way Python normally handles issues of code trying to use a protocol that
the object does not support is through *exceptions*.  When a 5000+ line
program produces meaningless results documentation not not very helpful to
start looking for the problem.  An exception gives you an approximate line
number and reason.

If __setitem__ on a tuple was ignored instead of producing an exception or
seek on a pipe failed silently I don't think that anyone would find "don't
do that, then" or "documentation" to be a satisfactory answer.

	Oren