[Python-Dev] namespace for generator expressions

[Tim Peters]

[Jeremy]
> Are there non-trivial examples?  The PEP suggests that they exist, but
> doesn't provide any.

Well, this example *should* be both easy and natural -- but it turns out to
be a disaster on two distinct counts if late binding is used:

http://mail.python.org/pipermail/python-dev/2003-October/039323.html

I'll repeat one bit from

http://mail.python.org/pipermail/python-dev/2003-October/039328.html

too:

    Whenever I've written a list-of-generators, or in the recent
    example a generator pipeline, I have found it semantically
    necessary, without exception so far, to capture the bindings of
    the variables whose bindings wouldn't otherwise be invariant
    across the life of the generator.  [If] it turns out that this
    is always, or nearly almost always, the case, across future
    examples too, then it would just be goofy not to implement
    generator expressions that way ("well, yes, the implementation
    does do a wrong thing in every example we had, but what you're
    not seeing is that the explanation would have been a line longer
    had the implementation done a useful thing instead" <wink>).

> Note that Armin Rigo suggested a small change here a few weeks ago
> that makes the list comp and the gen expr behave the same way.  The
> range(i) part of the gen expr is evaluated at the point of
> definition.  The gen expr, thus, translates to:
>
> def f(it):
>     for x in it:
>         yield x * 2
>
> iterators.append(f(range(i))
>
> As a result of this change, the only new scope is for the body of the
> target expression.  I don't know how that effects the earlier
> examples.

The example in the first link above is:

    pipe = source
    for p in predicates:
        # add a filter over the current pipe, and call that the new pipe
        pipe = e for e in pipe if p(e)

"p" and "pipe" both vary, and disaster ensues if the bindings (for both) in
effect at the time of each binding (to "pipe") aren't used when the last
generator in the chain (the final binding of "pipe") is provoked into
delivering results.  If Armin's suggestion transforms that to

    pipe = source
    for p in predicates:
        def g(it):
            for e in it:
                if p(e):
                    yield e
        pipe = g(pipe)

then the binding of "p" is left bound to predicates[-1] in all the
intermediate generators.

> BTW is there good terminology for describing the parts of a list comp
> or gen expr?  How about the iterator, the conditions, and the
> expression?

Probably more descriptive than body, mind and soul <wink>.  Fine by me!