Dangerous behavior of list(generator)

[Carl Banks]

On Dec 14, 7:21 am, exar... at twistedmatrix.com wrote:
> Note, I know *why* the implementation leads to this behavior.  I'm
> asking why "the devs" *accept* this.

As noted, the problem isn't with generators but with iteration
protocol.  The devs "allowed" this because it was a necessary evil for
correct functionality.  As the system is set up it can't discriminate
between a legitimate and a spurrious StopIteration.  (Which is why
Steven and others called you out for suggesting that the compiler has
to read your mind.)

However, as far as I'm concerned there is no reasonable argument that
this behavior is good.  So how, hypothetically, would one go about
fixing it, short of ripping out and replacing the existing machinery?

The first argument is that StopIteration has no place within a
generator expression.  Therefore a generator expression (but not a
generator function) could deliberately catch StopIteration and raise a
different exception.

I don't like it, though: who says that StopIteration has no place
within a generator expression?  Currently it's possible to do
something like this to terminate a genexp early, and I won't the one
saying you shouldn't do it.

def stop(): raise StopIteration

list(x or stop() for x in stream)

(Though personally I'd bite the bullet and write it as a generator
function).

What else?  The way I see it, when you throw StopIteration you are
trying to stop a specific generator.  Therefore StopIteration should
(somehow) contain a reference to the generator that it's being applied
to.  Perhaps it can be obtained by crawling ths stack, which shouldn't
be a significant penalty (it'd be only called once per iteration, and
most of the time it'd be only one or two frames up).  The looping
logic within Python should check whether the reference matches the
object it's iterating over; if not it raises a LeakyLoopException or
something like that.

I haven't thought this out though, I'm just kind of throwing this out
there.  Any issues?


But to answer your question, I think "simple is better than complex"
rules the day.  Right now StopIteration stops an iteration, simple as
that.  Any fix would add complexity.


Carl Banks