Re: [Python-ideas] A comprehension scope issue in PEP 572
[Tim]
- If the target is not local to any function enclosing F, and is not declared `global` in the block containing F, then the target is bound in the block containing F.
[also Tim]
FYI, that's still not right, ... I suspect that the above should be reworded to the simpler:
- If the target is not declared `global` or `nonlocal` in the block containing F, then the target is bound in the block containing F. ...
I'm satisfied that captures the intent - but now it's misleadingly wordy. It should be the briefer: - The target is bound in the block containing F. Other text (in section 4.2.2) already covers the intended meanings for when a `global` or `nonlocal` declaration appears in the block too. And then it's short enough again that the bullet list isn't really helpful anymore. So, putting that all together: """ An assignment expression binds the target, except in a function F synthesized to implement a list comprehension or generator expression (see XXX). In the latter case[1], the target is bound in the block containing F, and errors may be detected: If the target also appears as an identifier target of a `for` loop header in F, a `SyntaxError` exception is raised. If the block containing F is a class block, a `SyntaxError` exception is raised. Footnote: [1] The intent is that runtime binding of the target occurs as if the binding were performed in the block containing F. Because that necessarily makes the target not local in F, it's an error if the target also appears in a `for` loop header, which is a local binding for the same target. If the containing block is a class block, F has no access to that block's scope, so it doesn't make sense to consider the containing block. The target is bound in the containing block, where it inherits that block's `global` or `nonlocal` declaration if one exists, else establishes that the target is local to that block. """ Along with the previous:
... targets that are identifiers if occurring in an assignment, ...
" statement" should be inserted before the comma.
Of course there are other docs that need to change too. I viewed this as the hardest part (witness how long it took to reduce it to near-triviality ;-) ), so wanted to get it out of the way first.
On 13 May 2018 at 20:00, Tim Peters <tim.peters@gmail.com> wrote:
[Tim]
- If the target is not local to any function enclosing F, and is not declared `global` in the block containing F, then the target is bound in the block containing F.
[also Tim]
FYI, that's still not right, ... I suspect that the above should be reworded to the simpler:
- If the target is not declared `global` or `nonlocal` in the block containing F, then the target is bound in the block containing F. ...
I'm satisfied that captures the intent - but now it's misleadingly wordy. It should be the briefer:
- The target is bound in the block containing F.
Other text (in section 4.2.2) already covers the intended meanings for when a `global` or `nonlocal` declaration appears in the block too.
And then it's short enough again that the bullet list isn't really helpful anymore. So, putting that all together:
""" An assignment expression binds the target, except in a function F synthesized to implement a list comprehension or generator expression (see XXX). In the latter case[1], the target is bound in the block containing F, and errors may be detected: If the target also appears as an identifier target of a `for` loop header in F, a `SyntaxError` exception is raised. If the block containing F is a class block, a `SyntaxError` exception is raised.
Footnote: [1] The intent is that runtime binding of the target occurs as if the binding were performed in the block containing F. Because that necessarily makes the target not local in F, it's an error if the target also appears in a `for` loop header, which is a local binding for the same target. If the containing block is a class block, F has no access to that block's scope, so it doesn't make sense to consider the containing block. The target is bound in the containing block, where it inherits that block's `global` or `nonlocal` declaration if one exists, else establishes that the target is local to that block. """
This is getting pretty close to being precise enough to be at least potentially implementable (thanks!), but there are still two cases that would need to be covered: - what happens inside a lambda expression? - what happens inside another comprehension or generator expression? Cheers, Nick. -- Nick Coghlan | ncoghlan@gmail.com | Brisbane, Australia
[Tim]
""" An assignment expression binds the target, except in a function F synthesized to implement a list comprehension or generator expression (see XXX). In the latter case[1], the target is bound in the block containing F, and errors may be detected: If the target also appears as an identifier target of a `for` loop header in F, a `SyntaxError` exception is raised. If the block containing F is a class block, a `SyntaxError` exception is raised.
Footnote: [1] The intent is that runtime binding of the target occurs as if the binding were performed in the block containing F. Because that necessarily makes the target not local in F, it's an error if the target also appears in a `for` loop header, which is a local binding for the same target. If the containing block is a class block, F has no access to that block's scope, so it doesn't make sense to consider the containing block. The target is bound in the containing block, where it inherits that block's `global` or `nonlocal` declaration if one exists, else establishes that the target is local to that block. """
[Nick]
This is getting pretty close to being precise enough to be at least potentially implementable (thanks!), but there are still two cases that would need to be covered:
- what happens inside a lambda expression?
Section 4 of the Reference Manual doesn't contain the word "lambda", because there's no need to. "lambda" is just another way to create a function, and the behavior of functions is already specified. If you disagree, you mean something by "lambda expression" other than what I take it to mean, and the best way to illustrate what you do mean would be via giving a concrete example. As far as I'm concerned, ... (lambda...: expression) ... is exactly the same as def _hidden_unique_name(...): return expression ... (_hidden_unique_name) .... Even at class scope ;-) For example, def f(): g = lambda n: [(n := n+1) for i in range(1)] return g(10) is the same as: def f(): def g(n): return [(n := n+1) for i in range(1)] return g(10) When the listcomp synthesizes a function, g's code block will immediately contain it. The text at the top says `n` is bound in the containing block - which is g's. `n` is already local to `g`, so that part is just redundant in this case. The synthetic function will take 10 (via its nonlocal cell), add 1, and again via the cell rebind g's `n` to 11. The synthetic function returns [11] and the rebound `n` vanishes with its execution frame. But that's all about how functions behave; "lambda" is just incidental.
- what happens inside another comprehension or generator expression?
I don't know what you're asking about, so I'll just copy part of a different reply: """ Where are the docs explaining how nested comprehensions work today? I haven't seen any. If any such exist, I'd bet nothing about them needs to be changed. If none such exist, I don't see a need to write any just for this PEP. How do nested expressions of any kind work? Same thing. The only thing the suggestion changes is the scope of assignment expression targets in synthetic functions created to implement comprehensions. That has nothing at all to do with the possibility of nesting, or with the structure of nesting. Why do you think it does - or might? """ Note that a day or two ago I posted a complete expansion of what list(i + sum((i := i+1) for j in range(i)) + i for i in range(5)) would do. There the inner genexp rebinds the outer genexp's local for-target. Atrocious. Here's how you can do the same: Replace `(i := i+1)` with `(LATER)`. Generate the complete expansion for how the assignment-expression-free derived statement is implemented, presumably following the detailed docs that don't appear to exist ;-) for how that's done. In the synthetic function created for the inner genexp, add nonlocal i at the top and replace yield (LATER) with yield (i := i+1) Now I didn't replace anything with "LATER", because I never thought adding a new binary operator had anything to do with this process to begin with ;-) All that's needed is to add cruft _after_ it's done to establish the intended scopes for assignment statement targets. If you ask how the inner genexp "knows" that it needs to access the outer genexp's `i`, it doesn't directly. It's simply that the synthetic inner genexp function is nested inside the synthetic outer genexp function, and the text at top says that the inner genexp binds `i` in its containing block - which is the block for the synthetic outer genexp. If the functions synthesized for nested comprehensions don't _already_ nest in this way, then they'd already screw up merely accessing outer comprehension names from within inner comprehensions. Is there a reason to suspect that there's anything inherently unique to that specific example? I did a bunch of these "by hand", _trying_ to create problems, but didn't manage to. The only times I got even slightly flustered were when brain fog temporarily blocked my ability to see how to generate the nested messes _entirely independent_ of that they happened to contain assignment expressions. As I also posted about earlier, the real problems I've seen were in corner cases _without_ the suggested change, stemming from that different pieces of a comprehension execute in different scopes. That can have ugly consequences when an assignment expression appears in the outermost for's iterable, and its target is also in the body of the comprehension. That doesn't even require nesting to become incoherent. [y for _ in range(y := 42)] [y for y in range(y := 42)] With the suggestion, a binding expression target resolves to the same scope regardless of where it appears in the genexp/listcomp, so that class of head-scratcher vanishes ("same scope as in the containing block" implies that all instances of the target resolve to the same scope, which, happily enough, is resolved in the very block the outermost iterable _is_ executed in; so in the first example above both instances of `y` are resolved in the block containing the listcomp, and the second example is a compile-time error for the coherent reason given in the text at the top).
participants (2)
-
Nick Coghlan -
Tim Peters