<div dir="ltr"><div>Note that the weird, Action At A Distance behavior is also visible for locals() called at module scope (since there, locals() is globals(), which returns the actual dict that's the module's __dict__, i.e. the Source Of Truth. So I think it's unavoidable in general, and we would do wise not to try and "fix" it just for function locals. (And I certainly don't want to mess with globals().)</div><div><br></div><div>This is another case for the proposed [proxy] semantics, assuming we can get over our worry about backwards incompatibility.<br></div></div><br><div class="gmail_quote"><div dir="ltr" class="gmail_attr">On Mon, May 27, 2019 at 7:31 PM Steven D'Aprano <<a href="mailto:steve@pearwood.info">steve@pearwood.info</a>> wrote:<br></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex">On Mon, May 27, 2019 at 08:15:01AM -0700, Nathaniel Smith wrote:<br>
[...]<br>
> I'm not as sure about the locals() parts of the proposal. It might be<br>
> fine, but there are some complex trade-offs here that I'm still trying<br>
> to wrap my head around. The rest of this document is me thinking out<br>
> loud to try to clarify these issues.<br>
<br>
Wow. Thanks for the detail on this, I think the PEP should link to this <br>
thread, you've done some great work here.<br>
<br>
<br>
[...]<br>
> In function scopes, things are more complicated. The *local<br>
> environment* is conceptually well-defined, and includes:<br>
> - local variables (current source of truth: "fast locals" array)<br>
> - closed-over variables (current source of truth: cell objects)<br>
<br>
I don't think closed-over variables are *local* variables. They're <br>
"nonlocal", and you need a special keyword to write to them.<br>
<br>
<br>
> - any arbitrary key/values written to frame.f_locals that don't<br>
> correspond to local or closed-over variables, e.g. you can do<br>
> frame.f_locals[object()] = 10, and then later read it out again.<br>
<br>
Today I learned something new.<br>
<br>
<br>
> However, the mapping returned by locals() does not directly reflect<br>
> this local environment. Instead, each function frame has a dict<br>
> associated with it. locals() returns this dict. The dict always holds<br>
> any non-local/non-closed-over variables, and also, in certain<br>
> circumstances, we write a snapshot of local and closed-over variables<br>
> back into the dict.<br>
<br>
I'm going to try to make a case for your "snapshot" scenario.<br>
<br>
The locals dict inside a function is rather weird:<br>
<br>
- unlike in the global or class scope, writing to the dict does not <br>
update the variables<br>
<br>
- writing to it is discouraged, but its not a read-only proxy<br>
<br>
- it seems to be a snapshot of the state of variables when you <br>
called locals():<br>
<br>
# inside a function<br>
x = 1<br>
d = locals()<br>
x = 2<br>
assert d['x'] == 1<br>
<br>
- but it's not a proper, static, snapshot, because sometimes it<br>
will mutate without you touching it.<br>
<br>
That last point is Action At A Distance, and while it is explicable <br>
("there's only one locals dict, and calling locals() updates it") its <br>
also rather unintuitive and surprising and violates the Principle Of <br>
Least Surprise.<br>
<br>
[Usual disclaimers about "surprising to whom?" applies.]<br>
<br>
Unless I missed something, it doesn't seem that any of the code you <br>
(Nathan) analysed is making use of this AAAD behaviour, at least not <br>
deliberately. At least one of the examples took steps to avoid it by <br>
making an explicit copy after calling locals(), but missed one leaving <br>
that function possibly buggy.<br>
<br>
Given how weirdly the locals dict behaves, and how tricky it is to <br>
explain all the corner cases, I'm going to +1 your "snapshot" idea: <br>
<br>
- we keep the current behaviour for locals() in the global and class <br>
scopes;<br>
<br>
- we keep the PEP's behaviour for writebacks when locals() or exec()<br>
(and eval with walrus operator) are called, for the frame dict;<br>
<br>
- but we change locals() to return a copy of that dict, rather than<br>
the dict itself.<br>
<br>
(I think I've got the details right... please correct me if I've <br>
misunderstood anything.)<br>
<br>
Being a backwards-incompatible change, that means that folks who were <br>
relying on that automagical refresh of the snapshot will need to change <br>
their code to explicitly refresh:<br>
<br>
# update in place:<br>
d.update(locals())<br>
<br>
# or get a new snapshot<br>
d = locals()<br>
<br>
<br>
Or they explicitly grab a reference to the frame dict instead of <br>
calling locals(). Either way is likely to be less surprising than the <br>
status quo and less likely to lead to accidental, unexpected updates of <br>
the local dictionary without your knowledge.<br>
<br>
<br>
[...]<br>
> Of course, many of these edge cases are pretty obscure, so it's not<br>
> clear how much they matter. But I think we can at least agree that<br>
> this isn't the one obvious way to do it :-).<br>
<br>
Indeed. I thought I was doing well to know that writing to locals() <br>
inside a function didn't necessarily update the variable, but I had no <br>
idea of the levels of complexity actually involved!<br>
<br>
<br>
> I can think of a lot of criteria that all-else-being-equal we would<br>
> like Python to meet. (Of course, in practice they conflict.)<br>
> <br>
> Consistency across APIs: it's surprising if locals() and<br>
> frame.f_locals do different things. This argues for [proxy].<br>
<br>
I don't think it is that surprising, since frame.f_locals is kinda <br>
obscure (the average Python coder wouldn't know a frame if one fell <br>
on them) and locals() has been documented as weird for decades.<br>
<br>
In any case, at least "its a copy of ..." is simple and understandable.<br>
<br>
<br>
> Consistency across contexts: it's surprising if locals() has acts<br>
> differently in module/class scope versus function scope. This argues<br>
> for [proxy].<br>
<br>
True as far as it goes, but its also true that for the longest time, in <br>
most implementations, locals() has acted differently. So no change there.<br>
<br>
On the other hand, locals() currently returns a dict everywhere. It <br>
might be surprising for it to start returning a proxy object inside <br>
functions instead of a dict.<br>
<br>
<br>
-- <br>
Steven<br>
_______________________________________________<br>
Python-Dev mailing list<br>
<a href="mailto:Python-Dev@python.org" target="_blank">Python-Dev@python.org</a><br>
<a href="https://mail.python.org/mailman/listinfo/python-dev" rel="noreferrer" target="_blank">https://mail.python.org/mailman/listinfo/python-dev</a><br>
Unsubscribe: <a href="https://mail.python.org/mailman/options/python-dev/guido%40python.org" rel="noreferrer" target="_blank">https://mail.python.org/mailman/options/python-dev/guido%40python.org</a><br>
</blockquote></div><br clear="all"><br>-- <br><div dir="ltr" class="gmail_signature"><div dir="ltr"><div>--Guido van Rossum (<a href="http://python.org/~guido" target="_blank">python.org/~guido</a>)</div><div><i style="font-family:Arial,Helvetica,sans-serif;font-size:small;font-weight:400;letter-spacing:normal;text-align:start;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px;background-color:rgb(255,255,255);color:rgb(136,136,136)"><span>Pronouns</span>: he/him/his </i><a href="http://feministing.com/2015/02/03/how-using-they-as-a-singular-pronoun-can-change-the-world/" style="color:rgb(17,85,204);font-family:Arial,Helvetica,sans-serif;font-size:small;font-style:normal;font-weight:400;letter-spacing:normal;text-align:start;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px;background-color:rgb(255,255,255)" target="_blank"><i>(why is my <span>pronoun</span> here?)</i></a></div></div></div>