[Python-ideas] A comprehension scope issue in PEP 572
tim.peters at gmail.com
Sun May 6 23:46:00 EDT 2018
>> I have a long history of arguing that magically created lexically
>> nested anonymous functions try too hard to behave exactly like
>> explicitly typed lexically nested functions, but that's the trendy
>> thing to do so I always lose ;-)
[Nick Coghlan <ncoghlan at gmail.com>]
> You have the reasoning there backwards:
That's easy to believe - I also had a long history of resisting nested
scopes at all ;-)
> implicitly nested scopes behave like explicitly nested scopes because
> that was the *easy* way for me to implement them in Python 3.0 (since
> I got to re-use all the pre-existing compile time and runtime machinery
> that was built to handle explicit lexical scopes).
> Everything else I tried (including any suggestions made by others on the
> py3k mailing list when I discussed the problems I was encountering) ran into
> weird corner cases at either compile time or run time, so I eventually gave
> up and proposed that the implicit scope using to hide the iteration variable
> name binding be a full nested closure, and we'd just live with the
> consequences of that.
It's unfortunate that there are "consequences" at all. That kind of
thing is done all the time in Lisp-ish languages, but they require
explicitly declaring names' scopes. Python's "infer scope instead by
looking for bindings" worked great when it had 3 scopes total, but
keeps forcing "consequences" that may or may not be desired in a
> The sublocal scoping proposal in the earlier drafts of PEP 572 was our first
> serious attempt at defining a different way of doing things that would allow
> names to be hidden from surrounding code while still being visible in nested
> suites, and it broke people's brains to the point where Guido explicitly
> asked Chris to take it out of the PEP :)
To which removal I was sympathetic, BTW.
> However, something I *have* been wondering is whether or not it might make
> sense to allow inline scoping declarations in comprehension name bindings.
> Then your example could be written:
> def ...:
> p = None
> while any(n % p for nonlocal p in small_primes):
> # p was declared as nonlocal in the nested scope, so our p
> points to the last bound value
Which more directly addresses the underlying problem: not really
"binding expressions" per se, but the lack of control over scope
decisions in comprehensions period. It's not at all that nested
scopes are a poor model, it's that we have no control over what's
local _to_ nested scopes the language creates. I'd say that's worth
addressing in its own right, regardless of PEP 572's fate.
BTW, the "p = None" there is annoying too ;-)
> Needing to switch from "nonlocal p" to "global p" at module level would
> likely be slightly annoying, but also a reminder that the bound name is now
> visible as a module attribute.
Or `nonlocal` could be taught that its use one level below `global`
has an obvious meaning: global.
> If any other form of comprehension level name binding does eventually get
> accepted, then inline scope declarations could similarly be used to hoist
> values out into the surrounding scope:
> rem = None
> while any((nonlocal rem := n % p) for nonlocal p in small_primes):
> # p and rem were declared as nonlocal in the nested scope, so
> our rem and p point to the last bound value
Right - as above, inline scope declarations would be applicable to all
forms of comprehension-generated code. And to any other future
construct that creates lexically nested functions.
More information about the Python-ideas