<div dir="ltr">Just a quickie:<br><div class="gmail_quote"><div dir="ltr"><br>[Steve Dower]</div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"></blockquote>> The PEP uses the phrase "an assignment expression</div><div class="gmail_quote">> occurs in a comprehension" - what does this mean?<br><br>It's about static analysis of the source code, at compile-time, to establish scopes. So "occurs in" means your eyeballs see an assignment expression in a comprehension.</div><div class="gmail_quote"><br>> Does it occur when/where it is compiled, instantiated, or executed? This</div><div class="gmail_quote">> is important because where it occurs determines which scope will be<br>> modified. For sanity sake, I want to assume that it means compiled,<br><br>Probably ;-) I just don't know _exactly_ what those distinctions mean to you.<br><br>> but now what happens when that scope is gone?<br><br>Nothing new. There's no new concept of "scope" here, which is why the PEP doesn't say a whole lot about scope.<br><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex">
</blockquote><br><blockquote class="gmail_quote" style="margin:0px 0.8ex;border-left:1px solid rgb(204,204,204);border-right:1px solid rgb(204,204,204);padding-left:1ex;padding-right:1ex"></blockquote>> >>> def f():<br><blockquote class="gmail_quote" style="margin:0px 0.8ex;border-left:1px solid rgb(204,204,204);border-right:1px solid rgb(204,204,204);padding-left:1ex;padding-right:1ex"></blockquote>> ... return (a := i for i in range(5))<br><blockquote class="gmail_quote" style="margin:0px 0.8ex;border-left:1px solid rgb(204,204,204);border-right:1px solid rgb(204,204,204);padding-left:1ex;padding-right:1ex"></blockquote>> ...<br><br>Same as now, `i` is local to the synthetic nested function created for the genexp. The scope of `a` is determined by pretending the assignment occurred in the block containing the outermost (textually - static analysis) comprehension. In this case, `a = anything` before the `return` would establish that `a` is local to `f`, so that's the answer: `a` is local to `f`. If `a` had been declared global in `f`, then `a` in the genexp would be the same global `a`. And similarly if `a` had been declared <a href="http://nonlocal.in">nonlocal.in</a> `f`.<br><br>In all cases the scope resolution is inherited from the closest containing non-comprehension/genexp block, with the twist if that if a name is unknown in that block, the name is established as being local to that block. So your example is actually the subtlest case.</div><div class="gmail_quote"><br><br><blockquote class="gmail_quote" style="margin:0px 0.8ex;border-left:1px solid rgb(204,204,204);border-right:1px solid rgb(204,204,204);padding-left:1ex;padding-right:1ex"></blockquote>> >>> list(f())<br><blockquote class="gmail_quote" style="margin:0px 0.8ex;border-left:1px solid rgb(204,204,204);border-right:1px solid rgb(204,204,204);padding-left:1ex;padding-right:1ex"></blockquote>> [0, 1, 2, 3, 4] # or a new error because the scope has gone?<br><br>Function scopes in Python have "indefinite extent", and nothing about that changes. So, ya, that's the output - same as today if you changed your example to delete the "a :=" part.<br><br>Internally, f's local `a` was left bound to 4, but there's no way to see that here because the genexp has been run to exhaustion and reference-counting has presumably thrown everything away by now.<br><br><blockquote class="gmail_quote" style="margin:0px 0.8ex;border-left:1px solid rgb(204,204,204);border-right:1px solid rgb(204,204,204);padding-left:1ex;padding-right:1ex"></blockquote>>>> a<br><blockquote class="gmail_quote" style="margin:0px 0.8ex;border-left:1px solid rgb(204,204,204);border-right:1px solid rgb(204,204,204);padding-left:1ex;padding-right:1ex"></blockquote>???<br><br>Same thing typing `a` would result in if you had never typed `list(f())`.<br><br>Here's a variation:<br><br><span style="background-color:rgb(255,255,255);text-decoration-style:initial;text-decoration-color:initial;float:none;display:inline"> def f():</span><br style="background-color:rgb(255,255,255);text-decoration-style:initial;text-decoration-color:initial"><blockquote class="gmail_quote" style="background-color:rgb(255,255,255);text-decoration-style:initial;text-decoration-color:initial;margin:0px 0.8ex;border-left:1px solid rgb(204,204,204);border-right:1px solid rgb(204,204,204);padding-left:1ex;padding-right:1ex"></blockquote><span style="background-color:rgb(255,255,255);text-decoration-style:initial;text-decoration-color:initial;float:none;display:inline"> yield (a := i for i in range(5))</span></div><div class="gmail_quote"><span style="background-color:rgb(255,255,255);text-decoration-style:initial;text-decoration-color:initial;float:none;display:inline"> yield a</span></div><div class="gmail_quote"><span style="background-color:rgb(255,255,255);text-decoration-style:initial;text-decoration-color:initial;float:none;display:inline"><br></span></div><div class="gmail_quote"><span style="background-color:rgb(255,255,255);text-decoration-style:initial;text-decoration-color:initial;float:none;display:inline">Then:<br><br>>>> g = f()</span></div><div class="gmail_quote"><span style="background-color:rgb(255,255,255);text-decoration-style:initial;text-decoration-color:initial;float:none;display:inline">>>> list(next(g))</span></div><div class="gmail_quote"><span style="background-color:rgb(255,255,255);text-decoration-style:initial;text-decoration-color:initial;float:none;display:inline">[0, 1, 2, 3, 4]</span></div><div class="gmail_quote"><span style="background-color:rgb(255,255,255);text-decoration-style:initial;text-decoration-color:initial;float:none;display:inline">>>> next(g)</span></div><div class="gmail_quote"><span style="background-color:rgb(255,255,255);text-decoration-style:initial;text-decoration-color:initial;float:none;display:inline">4</span></div><div class="gmail_quote"><span style="background-color:rgb(255,255,255);text-decoration-style:initial;text-decoration-color:initial;float:none;display:inline"><br></span></div><div class="gmail_quote"><blockquote class="gmail_quote" style="background-color:rgb(255,255,255);text-decoration-style:initial;text-decoration-color:initial;margin:0px 0.8ex;border-left:1px solid rgb(204,204,204);border-right:1px solid rgb(204,204,204);padding-left:1ex;padding-right:1ex"></blockquote><span style="background-color:rgb(255,255,255);text-decoration-style:initial;text-decoration-color:initial;float:none;display:inline"></span>> I'll push back real hard on doing the assignment in the scope</div><div class="gmail_quote">> where the generator is executed:<br><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex">
</blockquote>> <br><blockquote class="gmail_quote" style="margin:0px 0.8ex;border-left:1px solid rgb(204,204,204);border-right:1px solid rgb(204,204,204);padding-left:1ex;padding-right:1ex"></blockquote>> >>> def do_secure_op(name, numbers):<br><blockquote class="gmail_quote" style="margin:0px 0.8ex;border-left:1px solid rgb(204,204,204);border-right:1px solid rgb(204,204,204);padding-left:1ex;padding-right:1ex"></blockquote>> ... authorised = check_authorised(name)<br><br>This instance of `authorized` is local to `do_secure_op`.<br><br><blockquote class="gmail_quote" style="margin:0px 0.8ex;border-left:1px solid rgb(204,204,204);border-right:1px solid rgb(204,204,204);padding-left:1ex;padding-right:1ex"></blockquote>> ... if not all(numbers):<br><blockquote class="gmail_quote" style="margin:0px 0.8ex;border-left:1px solid rgb(204,204,204);border-right:1px solid rgb(204,204,204);padding-left:1ex;padding-right:1ex"></blockquote>> ... raise ValueError()<br><blockquote class="gmail_quote" style="margin:0px 0.8ex;border-left:1px solid rgb(204,204,204);border-right:1px solid rgb(204,204,204);padding-left:1ex;padding-right:1ex"></blockquote>> ... if not authorised:<br><blockquote class="gmail_quote" style="margin:0px 0.8ex;border-left:1px solid rgb(204,204,204);border-right:1px solid rgb(204,204,204);padding-left:1ex;padding-right:1ex"></blockquote>> ... raise SecurityError()<br><blockquote class="gmail_quote" style="margin:0px 0.8ex;border-left:1px solid rgb(204,204,204);border-right:1px solid rgb(204,204,204);padding-left:1ex;padding-right:1ex"></blockquote>> ... print('You made it!')<br><blockquote class="gmail_quote" style="margin:0px 0.8ex;border-left:1px solid rgb(204,204,204);border-right:1px solid rgb(204,204,204);padding-left:1ex;padding-right:1ex"></blockquote>> ...<br><blockquote class="gmail_quote" style="margin:0px 0.8ex;border-left:1px solid rgb(204,204,204);border-right:1px solid rgb(204,204,204);padding-left:1ex;padding-right:1ex"></blockquote>> >>> do_secure_op('whatever', (authorised := i for i in [1, 2, 3]))</div><div class="gmail_quote"><br></div><div class="gmail_quote">And this instance of `authorized` is a global (because the genexp appears in top-level code, so its containing block is the module). The two instances have nothing to do with each other.<br><br></div><div class="gmail_quote"><blockquote class="gmail_quote" style="margin:0px 0.8ex;border-left:1px solid rgb(204,204,204);border-right:1px solid rgb(204,204,204);padding-left:1ex;padding-right:1ex"></blockquote>> You made it!<br><br>Yup - you did!</div><div class="gmail_quote"><br><blockquote class="gmail_quote" style="margin:0px 0.8ex;border-left:1px solid rgb(204,204,204);border-right:1px solid rgb(204,204,204);padding-left:1ex;padding-right:1ex"></blockquote>> >>> authorised<br><blockquote class="gmail_quote" style="margin:0px 0.8ex;border-left:1px solid rgb(204,204,204);border-right:1px solid rgb(204,204,204);padding-left:1ex;padding-right:1ex"></blockquote>> NameError: name 'authorised' is undefined<br><br>It would display 3 instead.<br><br><br><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"></blockquote>> From the any()/all() examples, it seems clear that the target scope for<br><blockquote class="gmail_quote" style="margin:0px 0.8ex;border-left:1px solid rgb(204,204,204);border-right:1px solid rgb(204,204,204);padding-left:1ex;padding-right:1ex"></blockquote>> the assignment has to be referenced from the generator scope (but not<br><blockquote class="gmail_quote" style="margin:0px 0.8ex;border-left:1px solid rgb(204,204,204);border-right:1px solid rgb(204,204,204);padding-left:1ex;padding-right:1ex"></blockquote>> for other comprehension types, which can simply do one transfer of the<br><blockquote class="gmail_quote" style="margin:0px 0.8ex;border-left:1px solid rgb(204,204,204);border-right:1px solid rgb(204,204,204);padding-left:1ex;padding-right:1ex"></blockquote>> assigned name after fully evaluating all the contents).<br><br>I don't think that follows. It _may_ in some cases. For example,<br><br><div class="gmail_quote">def f():</div><div class="gmail_quote"> i = None # not necessary, just making i's scope obvious</div><div class="gmail_quote"> def g(ignore):</div><div class="gmail_quote"> return i+1</div><div class="gmail_quote"> return [g(i := j) for j in range(3)]</div><div><br>_While_ the list comprehension is executing, it needs to rebind f's `i` on each iteration so that the call to `g()` on each iteration can see `i`'s then-current value.</div><div><br></div>> Will this reference keep the frame object alive for as</div><div class="gmail_quote">> long as the generator exists? Can it be a weak reference?<br>> Are assignments just going to be silently ignored when<br>> the frame they should assign to is gone? I'd like to see these<br>> clarified in the main text.<br><br>Here you're potentially asking questions about how closures work in Python (in the most-likely case that an embedded assignment-statement target resolves to an enclosing function-local scope), but the PEP doesn't change anything about how closures already work. Closures are implemented via "cell objects", one per name, which already supply both "read" and "write" access to both the owning and referencing scopes.</div><div class="gmail_quote"><br></div><div class="gmail_quote"><div class="gmail_quote">def f():</div><div class="gmail_quote"> a = 42</div><div class="gmail_quote"> return (a+1 for i in range(3))</div><div><br>That works fine today, and a cell object is used in the synthesized genexp function to get read access to f's local `a`. But references to `a` in `f` _also_ use that cell object. - the thing that lives in f's frame isn't really the binding for `a`, but a reference to the cell object that _holds_ a's current binding. The cell object doesn't care whether f's frame goes away (except that the cell object's refcount drops by one when f's frame vanishes). Nothing about that changes if the synthesized genexp wants write access instead.<br><br>While a gentle introduction to how closures are implemented in Python would be a cool thing, this PEP is the last place to include one ;-)<br><br></div><div>It may help to realize that there's nothing here that can't be done today by explicitly writing nested functions with appropriate scope declarations, in a straightforward way. There's nothing _inherently_ new.<br></div></div><div class="gmail_quote"><br>Huh! Not so much "a quickie" after all :-( So I'll stop here for now. Thank you for the critical reading and feedback!<br><br></div></div>