On Sun, Nov 2, 2014 at 1:00 PM, Terry Reedy <tjreedy@udel.edu> wrote:
On 11/2/2014 2:50 PM, Andrew Barnert wrote:
This is related to the fact that, although the docs imply otherwise,
[COMP] isn't exactly equivalent to list(COMP),

That purported equivalence is a common meme, which I may have helped spread.

I may have started it. I was aware of the non-equivalence (only mostly-equivalence) in Python 2 and I wanted to make then identical in Python 3 -- having one construct being exactly equivalent to another reduce the amount of explaining needed. Unfortunately, people had started to depend on the (in my *current* opinion deplorable) behavior of generator expressions in the face of StopIteration thrown by arbitrary parts of the expression or condition, and the equivalence is still imperfect. At least the variable leakage has been fixed.

I know that when we first introduced generators (not generator expressions) I was in favor of interpreting arbitrary things that raise StopIteration in a generator to cause the generator to terminate just as if it had decided to stop (i.e. 'return' or falling of the end), because I thought there were some useful patterns that could be written more compactly this way -- in particular, the pattern where a generator iterates over another iterator by calling next() on it, does some processing on the value thus produced, and then yielding the processed value (or not), and where the logical response to a StopIteration from the inner iterator is to exit the generator. For example:

  def only_positive(it):
      while True:
          x = next(it)
          if x > 0: yield x

This *particular* example is much better written as:

  def only_positive(x):
      for x in it:
          if x > 0: yield x

but the idea was that there might be variants where being constrained by a single for-loop would make the code less elegant if you had to catch the StopIteration and explicitly exit the generator.

However, I don't think this idea has panned out. I haven't done a survey, but I have a feeling that in most cases where an explicit next() call is used (as opposed to a for-loop) there's a try/except Stopiteration around it, and a fair amount if time is wasted debugging situations where a StopIteration unexpectedly escapes and silently interrupts some loop over an unrelated generator (instead of loudly bubbling up to the top and causing a traceback, which would be more debuggable). And the use case of raising StopIteration from a condition used in a generator expression is iffy at best (it makes the condition function hard to use in other contexts, and it calls to attention the difference between generators and comprehensions).

So I will go out on a limb here and say that this was a mistake and if we can think of easing the transitional pain it would be a good thing to fix this eventually.

--
--Guido van Rossum (python.org/~guido)