[Python-Dev] Single- vs. Multi-pass iterability

[Ka-Ping Yee]

On Thu, 18 Jul 2002, Guido van Rossum wrote:
> First of all, I'm not sure what exactly the issue is with destructive
> for-loops.

It's just not the way i expect for-loops to work.  Perhaps we would
need to survey people for objective data, but i feel that most people
would be surprised if

    for x in y: print x
    for x in y: print x

did not print the same thing twice, or if

    if x in y: print 'got it'
    if x in y: print 'got it'

did not do the same thing twice.  I realize this is my own opinion,
but it's a fairly strong impression i have.

Even if it's okay for for-loops to destroy their arguments, i still
think it sets up a bad situation: we may end up with functions
manipulating sequence-like things all over, but it becomes unclear
whether they destroy their arguments or not.  It becomes possible
to write a function which sometimes destroys its argument and sometimes
doesn't.  Bugs get deeper and harder to find.

I believe this is where the biggest debate lies: whether "for" should be
non-destructive.  I realize we are currently on the other side of the
fence, but i foresee enough potential pain that i would like you to
consider the value of keeping "for" loops non-destructive.

> Maybe the for-loop is a red herring?  Calling next() on an
> iterator may or may not be destructive on the underlying "sequence" --
> if it is a generator, for example, I would call it destructive.

Well, for a generator, there is no underlying sequence.

    while 1: print next(gen)

makes it clear that there is no sequence, but

    for x in gen: print x

seems to give me the impression that there is.

> Perhaps you're trying to assign properties to the iterator abstraction
> that aren't really there?

I'm assigning properties to "for" that you aren't.  I think they
are useful properties, though, and worth considering.

I don't think i'm assigning properties to the iterator abstraction;
i expect iterators to destroy themselves.  But the introduction of
iterators, in the way they are now, breaks this property of "for"
loops that i think used to hold almost all the time in Python, and
that i think holds all the time in almost all other languages.

> Next, I'm not sure how renaming next() to __next__() would affect the
> situation w.r.t. the destructivity of for-loops.  Or were you talking
> about some other migration?

The connection is indirect.  The renaming is related to: (a) making
__next__() a real, honest-to-goodness protocol independent of __iter__;
and (b) getting rid of __iter__ on iterators.  It's the presence of
__iter__ on iterators that breaks the non-destructive-for property.

I think the renaming of next() to __next__() is a good idea in any
case.  It is distant enough from the other issues that it can be done
independently of any decisions about __iter__.


-- ?!ng