[Python-ideas] A comprehension scope issue in PEP 572

[Guido van Rossum]

I am convinced by Tim's motivation. I hadn't thought of this use case
before -- I had mostly thought "local scope in a comprehension or generator
expression is the locals of the synthetic function". But Tim's reasoning
feels right.

The only solution that makes sense to me is Steven's (2). (1) is what the
PEP currently says and what Tim doesn't want; (3) has no precedent
(function defaults don't really work like this) and just gets my hackles
all up. (I can't even tell if Steven meant it as a serious proposal.)

So let's see if we can change PEP 572 so that := inside a comprehension or
generator expression al ways assigns to a variable in the containing scope.

It may be inconsistent with the scope of the loop control variable, but
it's consistent with uses of := outside comprehensions:

  [x := 0]
  [x := i for i in range(1)]

both have the side effect of setting x to zero. I like that.

There's one corner case (again) -- class scopes. If the containing scope is
a function, everything's fine, we can use the existing closure machinery.
If the containing scope is global, everything's fine too, we can treat it
as a global. But if the containing scope is a class, we can't easily extend
the current machinery. But this breakage is similar to the existing
breakage with comprehensions in class scope that reference class variables:

  class C:
      hosts = ['boring', 'haring', 'tering']
      full_hosts = [host + suffix for suffix in ('.cwi.nl', '.com') for
host in hosts]

Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "<stdin>", line 3, in C
  File "<stdin>", line 3, in <listcomp>
NameError: name 'hosts' is not defined

I propose to punt on this case. If we want to fix it we can fix it in a
number of ways and the fix can easily apply to both getting and setting --
but this is a separate fix (and we should take it out of PEP 572).

PS1. The various proposals that add random extra keywords to the syntax
(like 'for nonlocal i') don't appeal to me at all.

PS2. IIRC the reason we gave loop control variables their own scope was the
poor imagination of many people when it comes to choosing loop control
variable names. We had seen just too many examples of

  for x in something:
      ...lots of code using x...
      blah blah [x+1 for x in something else]
      ...some more code using x, broken...

It's true that this can also happen with a for-loop statement nested inside
the outer loop (and it does) but the case of a comprehension was easier to
miss. I've never looked back.

PS3. Comprehensions and generator expressions should be interchangeable.
They just look too similar to have different semantics (and the possibly
delayed execution of generator expressions is not an issue -- it's rare,
and closure semantics make it work just fine).

-- 
--Guido van Rossum (python.org/~guido)
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20180507/087a5382/attachment.html>