[Python-ideas] A comprehension scope issue in PEP 572
tjreedy at udel.edu
Mon May 7 16:35:54 EDT 2018
On 5/7/2018 1:38 PM, Guido van Rossum wrote:
> I am convinced by Tim's motivation. I hadn't thought of this use case
> before -- I had mostly thought "local scope in a comprehension or
> generator expression is the locals of the synthetic function". But Tim's
> reasoning feels right.
> The only solution that makes sense to me is Steven's (2). (1) is what
> the PEP currently says and what Tim doesn't want; (3) has no precedent
> (function defaults don't really work like this) and just gets my hackles
> all up. (I can't even tell if Steven meant it as a serious proposal.)
> So let's see if we can change PEP 572 so that := inside a comprehension
> or generator expression al ways assigns to a variable in the containing
> It may be inconsistent with the scope of the loop control variable, but
> it's consistent with uses of := outside comprehensions:
> [x := 0]
> [x := i for i in range(1)]
> both have the side effect of setting x to zero. I like that.
If I am understanding correctly, this would also let one *intentionally
'leak' (export) the last value of the loop variable when wanted.
[math.log(xlast:=x) for x in it if x > 0]
> There's one corner case (again) -- class scopes. If the containing scope
> is a function, everything's fine, we can use the existing closure
> machinery. If the containing scope is global, everything's fine too, we
> can treat it as a global. But if the containing scope is a class, we
> can't easily extend the current machinery. But this breakage is similar
> to the existing breakage with comprehensions in class scope that
> reference class variables:
> class C:
> hosts = ['boring', 'haring', 'tering']
> full_hosts = [host + suffix for suffix in ('.cwi.nl
> <http://cwi.nl>', '.com') for host in hosts]
> Traceback (most recent call last):
> File "<stdin>", line 1, in <module>
> File "<stdin>", line 3, in C
> File "<stdin>", line 3, in <listcomp>
> NameError: name 'hosts' is not defined
This is a special case of the fact that no function called in class
scope can access class variables, even if defined in the class scope.
>>> class C:
x = 0
z = f()
Traceback (most recent call last):
File "<pyshell#5>", line 1, in <module>
File "<pyshell#5>", line 5, in C
z = f()
File "<pyshell#5>", line 4, in f
NameError: name 'x' is not defined
I would find it strange if only functions defined by a comprehension
were given new class scope access.
> PS1. The various proposals that add random extra keywords to the syntax
> (like 'for nonlocal i') don't appeal to me at all.
> PS2. IIRC the reason we gave loop control variables their own scope was
> the poor imagination of many people when it comes to choosing loop
> control variable names. We had seen just too many examples of
> for x in something:
> ...lots of code using x...
> blah blah [x+1 for x in something else]
> ...some more code using x, broken...
> It's true that this can also happen with a for-loop statement nested
> inside the outer loop (and it does) but the case of a comprehension was
> easier to miss. I've never looked back.
> PS3. Comprehensions and generator expressions should be interchangeable.
> They just look too similar to have different semantics (and the possibly
> delayed execution of generator expressions is not an issue -- it's rare,
> and closure semantics make it work just fine).
To me, this is the prime justification for the 3.0 comprehension change.
I currently see a comprehension as a specialized generator expression.
A generator expression generalizes math set builder notation. If left
'raw', the implied function yields the values generated (what else could
it do?). If a collection type is indicated by the fences and expression
form, values are instead added to an anonymous instance thereof.
Terry Jan Reedy
More information about the Python-ideas