[Python-Dev] Informal educator feedback on PEP 572 (was Re: 2018 Python Language Summit coverage, last part)

[Guido van Rossum]

On Sun, Jun 24, 2018 at 12:03 PM Tim Peters <tim.peters at gmail.com> wrote:

> [Guido]
> :> However, [Tim] did post his motivation for (b) on python-ideas, IIRC a
> bit
> > before PyCon; and the main text of the PEP gives a strong motivation
> > (https://www.python.org/dev/peps/pep-0572/#scope-of-the-target).
> Nevertheless,
> > maybe we should compromise and drop (b)?
>
> Two things to say about that.  First, the original example I gave would be
> approximately as well addressed by allowing to declare intended scopes in
> magically synthesized functions; like (say)
>
> p = None # to establish the intended scope of `p`
> while any(<nonlocal p>  # split across lines just for readability
>                 n % p == 0 for p in small_primes):
>     n //= p
>
> It didn't really require an inline assignment, just a way to override the
> unwanted (in this case) "all `for` targets are local to the invisible
> function" rigid consequence of the implementation du jour.
>

Hm, that's more special syntax. The nice bit about (b) as currently
specified is that it adds no syntax -- it adds a scope rule, but (as IIRC
Steven has convincingly argued) few people care about those. Python's scope
rules, when fully specified, are intricate to the point of being arcane
(e.g. for class scopes) but all that has a purpose -- to make them so DWIM
("Do what I Mean") that in practice you almost never have to worry about
them, *especially* when reading non-obfuscated code (and also when writing,
except for a few well-known patterns).


> Second, if it's dropped, then the PEP needs more words to define what
> happens in cases like the following, because different textual parts of a
> comprehension execute in different scopes, and that can  become visible
> when bindings can be embedded:
>
> def f():
>     y = -1
>     ys = [y for _ in range(y := 5)]
>     print(y, ys)
>
> Here `range(y := 5)` is executed in f's scope.  Presumably the `y` in `y
> for` also refers to f's scope, despite that `y` textually _appears_ to be
> assigned to in the body of the listcomp, and so would - for that reason -
> expected to be local to the synthesized function, and so raise
> `UnboundLocalError` when referenced.  It's incoherent without detailed
> knowledge of the implementation.
>

That code should have the same meaning regardless of whether we accept (b)
or not -- there is only one `y`, in f's scope. I don't mind if we have to
add more words to the PEP's scope rules to make this explicit, though I
doubt it -- the existing weirdness (in the comprehension spec) about the
"outermost iterable" being evaluated in the surrounding scope specifies
this. I wouldn't call it incoherent -- I think what I said about scope
rules above applies here, it just does what you expect.


> def g():
>     y = -1
>     ys = [y for y in range(y := 5)]
>     print(y, ys)
>
> And here the `y` in `y for y` is local to the synthesized function, and
> presumably has nothing to do with the `y` in the `range()` call.  That's
> incoherent in its own way.
>
> Under the current PEP, all instances of `y` in `f` refer to the f-local
> `y`, and the listcomp in `g` is a compile-time error.
>

And under the (b)-less proposal, `g` would interpret `y for y` as both
referring to a new variable created just for the comprehension, and `y :=
5` as referring to g's scope. Again I don't think it needs extra words in
the spec. And the end user docs might just say "don't do that" (with a link
to the reference manual's rule about the "outermost iterable").

Even if in the end we did find a case where we'd have to write an explicit
rule to make what happens here a consequence of the spec rather than the
implementation, that doesn't count as an argument for keeping (b) to me.

In favor of (b) we have a few examples (see
https://www.python.org/dev/peps/pep-0572/#scope-of-the-target) that require
it, and more that you described on python-ideas (and also the motivating
use case from the thread that Steven dug up, starting here:
https://mail.python.org/pipermail/python-ideas/2018-April/049622.html).

A "neutral" argument about (b) is that despite the "horrified" reactions
that Nick saw, in practice it's going to confuse very few people (again,
due to my point about Python's scope rules). I'd wager that the people who
might be most horrified about it would be people who feel strongly that the
change to the comprehension scope rules in Python 3 is a big improvement,
and who are familiar with the difference in implementation of
comprehensions (though not generator expressions) in Python 2 vs. 3.

-- 
--Guido van Rossum (python.org/~guido)
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20180624/7233895c/attachment.html>