scope of generators, class variables, resulting in global na

[dontspamleo]

Hi Folks,

Thanks everyone for the great contributions! I understand this better
now. The distinction between a shorthand for a function definition and
a shorthand for a loop iteration is crucial.

Also: thanks for pointing out the even the list comprehension doesn't
work in py3. That was incredibly useful! I was about to build a
package using Python and now (unfortunately) I will have to find
another language. Saved me a big headache!

More details...

>
> I can't reproduce the error message that you cite.
>

Sorry, I made a cut and paste error in my first post. The error was
exactly the one in your post.

> Reason for the NameError:
>
> The above is a /generator expression/, evaluated in a class definition.
>
> The docs have a similar example but I'm sorry, I'm unable to find it! Anyway the
> generator expression is evaluated as if its code was put in function. And from
> within the scope of that function you can't access the class scope implicitly,
> hence, no access to 'B'.
>
<snip/>
>
> This is a /list comprehension/, not a generator expression (although
> syntactically it's almost the same).
>
> It obeys different rules.
>
> Essentially the generator expression produces a generator object that you may
> name or pass around as you wish, while the comprehension is just a syntactical
> device for writing more concisely some equivalent code that's generated inline.
>
> However, apparently the rules changed between Python 2.x and Python 3.x.
>
> In Python 3.x also the list comprehension fails in a class definition:
>
</snip>
> C:\test> py3
> Python 3.1.1 (r311:74483, Aug 17 2009, 17:02:12) [MSC v.1500 32 bit (Intel)] on
> win32
> Type "help", "copyright", "credits" or "license" for more information.
>  >>> class T:
> ...   A = range(2)
> ...   B = range(4)
> ...   s = sum([(i*j) for i in A for j in B])
> ...
> Traceback (most recent call last):
>    File "<stdin>", line 1, in <module>
>    File "<stdin>", line 4, in T
>    File "<stdin>", line 4, in <listcomp>
> NameError: global name 'B' is not defined
>  >>> exit()

Yuck! Why should the list comprehension fail in Py3? The scope rules
that explain why the generator expression would fail don't apply in
that case. Plus Guido's comment on generators being consumed quickly
also doesn't apply.

>
>  From one point of view it's good that Py3 provides about the same behavior for
> generator expressions and list comprehensions.
>
> But I'd really like the above examples to Just Work. :-)
>

Absolutely agreed. Especially with the list comprehension.

I read PEP 289 and the 2004 exchanges in the mailing list regarding
scoping and binding issues in generator expressions, when this feature
was added to the language. Forgive my blasphemy, but Guido got it
wrong on this one, by suggesting that an obscure use case should drive
design considerations when a far more common use case exists. The
obscure use case involves confusing sequences of exceptions, while the
common use case is consistent scope rules at different levels. For
details search the mailing list for Guido's quote in PEP 289.

Here is another example that fails...
In [7]: class T:
   ...:     A = range(2)
   ...:     B = range(2,4)
   ...:     g = (i*j for i in A for j in B)
   ...:     s = sum(g)
---------------------------------------------------------------------------
NameError                                 Traceback (most recent call
last)
C:\Python26\<ipython console> in <module>()
C:\Python26\<ipython console> in T()
C:\Python26\<ipython console> in <genexpr>((i,))
NameError: global name 'B' is not defined

... at sum(g)!

These two examples work (doing the same thing at a global scope and a
local scope):

In [1]: class T:
   ...:     def local(self):
   ...:         A = range(2)
   ...:         B = range(2,4)
   ...:         g = (i*j for i in A for j in B)
   ...:         s = sum(g)
   ...:
In [2]: t = T()
In [3]: t.local()
In [4]: A = range(2)
In [5]: B = range(2,4)
In [6]: g = (i*j for i in A for j in B)
In [7]: s = sum(g)

Thanks to everyone who suggested workarounds. They are very helpful.
At the same time, they are -- forgive my language -- so perlish (as in
clever, and requiring deep understanding of the language). In Python,
simple elegant constructs should work consistently across all scopes.
The very fact that people chose to use the word 'workaround' indicates
how quirky this aspect of the language is.

It really doesn't need to be so quirky. Defining the semantics of
generator expressions to mimic Arnaud's lambda workaround below would
be just as justified as the current definition, and more pythonic (in
the consistent, elegant sense). @Arnaud: I tried to find your earlier
post -- googled "Arnaud lambda" -- but couldn't.

Does anyone know of a more recent PEP that addresses these issues?

Thanks,
Leo.