Generator expressions vs. comprehensions

[Carl Banks]

On May 24, 3:31 pm, Ian Kelly <ian.g.ke... at gmail.com> wrote:
> Hi all,
>
> I just ran into an interesting but rather subtle little wart today.
> The following expressions are not functionally equivalent, even in
> Python 3:
>
> tuple(iterator.next() for i in xrange(n))
>
> tuple([iterator.next() for i in xrange(n)])
>
> In the first case, if iterator.next() raises a StopIteration, the
> exception is swallowed by the generator expression.  The expression
> evaluates to a truncated tuple, and the StopIteration is not
> propagated.
>
> In the second case, the StopIteration exception is propagated as
> expected by the list comprehension.  Set and dict comprehensions also
> behave this way in Python 3.
>
> Is this distinction generally known?  The generator expression
> behavior is understandable since a generator would do the same thing,
> but I'm disappointed that the inconsistency exists and wasn't fixed in
> Python 3 when we had the chance.


As a general rule you shouldn't call the next() method/function
without arranging to catch StopIteration, unless you know the iterator
will never raise StopIteration (such as it if it itertools.count()).
The problem is that if a StopIteration "leaks", as it does with the
generator examples, another iterator further up the stack might catch
it and exit.

The situation here is known.  It can't be corrected, even in Python 3,
without modifying iterator protocol to tie StopIteration to a specific
iterator.  This is possible and might be worth it to avoid hard-to-
diagnose bugs but it would complicate iterator protocol, which becomes
less useful as it becomes more complex.


Carl Banks