[Python-Dev] [PEP 558] thinking through locals() semantics

Tue May 28 00:17:23 EDT 2019

Note that the weird, Action At A Distance behavior is also visible for
locals() called at module scope (since there, locals() is globals(), which
returns the actual dict that's the module's __dict__, i.e. the Source Of
Truth. So I think it's unavoidable in general, and we would do wise not to
try and "fix" it just for function locals. (And I certainly don't want to
mess with globals().)

This is another case for the proposed [proxy] semantics, assuming we can
get over our worry about backwards incompatibility.

On Mon, May 27, 2019 at 7:31 PM Steven D'Aprano <steve at pearwood.info> wrote:

> On Mon, May 27, 2019 at 08:15:01AM -0700, Nathaniel Smith wrote:
> [...]
> > I'm not as sure about the locals() parts of the proposal. It might be
> > fine, but there are some complex trade-offs here that I'm still trying
> > to wrap my head around. The rest of this document is me thinking out
> > loud to try to clarify these issues.
>
> Wow. Thanks for the detail on this, I think the PEP should link to this
> thread, you've done some great work here.
>
>
> [...]
> > In function scopes, things are more complicated. The *local
> > environment* is conceptually well-defined, and includes:
> > - local variables (current source of truth: "fast locals" array)
> > - closed-over variables (current source of truth: cell objects)
>
> I don't think closed-over variables are *local* variables. They're
> "nonlocal", and you need a special keyword to write to them.
>
>
> > - any arbitrary key/values written to frame.f_locals that don't
> > correspond to local or closed-over variables, e.g. you can do
> > frame.f_locals[object()] = 10, and then later read it out again.
>
> Today I learned something new.
>
>
> > However, the mapping returned by locals() does not directly reflect
> > this local environment. Instead, each function frame has a dict
> > associated with it. locals() returns this dict. The dict always holds
> > any non-local/non-closed-over variables, and also, in certain
> > circumstances, we write a snapshot of local and closed-over variables
> > back into the dict.
>
> I'm going to try to make a case for your "snapshot" scenario.
>
> The locals dict inside a function is rather weird:
>
> - unlike in the global or class scope, writing to the dict does not
>   update the variables
>
> - writing to it is discouraged, but its not a read-only proxy
>
> - it seems to be a snapshot of the state of variables when you
>   called locals():
>
>       # inside a function
>       x = 1
>       d = locals()
>       x = 2
>       assert d['x'] == 1
>
> - but it's not a proper, static, snapshot, because sometimes it
>   will mutate without you touching it.
>
> That last point is Action At A Distance, and while it is explicable
> ("there's only one locals dict, and calling locals() updates it") its
> also rather unintuitive and surprising and violates the Principle Of
> Least Surprise.
>
> [Usual disclaimers about "surprising to whom?" applies.]
>
> Unless I missed something, it doesn't seem that any of the code you
> (Nathan) analysed is making use of this AAAD behaviour, at least not
> deliberately. At least one of the examples took steps to avoid it by
> making an explicit copy after calling locals(), but missed one leaving
> that function possibly buggy.
>
> Given how weirdly the locals dict behaves, and how tricky it is to
> explain all the corner cases, I'm going to +1 your "snapshot" idea:
>
> - we keep the current behaviour for locals() in the global and class
>   scopes;
>
> - we keep the PEP's behaviour for writebacks when locals() or exec()
>   (and eval with walrus operator) are called, for the frame dict;
>
> - but we change locals() to return a copy of that dict, rather than
>   the dict itself.
>
> (I think I've got the details right... please correct me if I've
> misunderstood anything.)
>
> Being a backwards-incompatible change, that means that folks who were
> relying on that automagical refresh of the snapshot will need to change
> their code to explicitly refresh:
>
>     # update in place:
>     d.update(locals())
>
>     # or get a new snapshot
>     d = locals()
>
>
> Or they explicitly grab a reference to the frame dict instead of
> calling locals(). Either way is likely to be less surprising than the
> status quo and less likely to lead to accidental, unexpected updates of
> the local dictionary without your knowledge.
>
>
> [...]
> > Of course, many of these edge cases are pretty obscure, so it's not
> > clear how much they matter. But I think we can at least agree that
> > this isn't the one obvious way to do it :-).
>
> Indeed. I thought I was doing well to know that writing to locals()
> inside a function didn't necessarily update the variable, but I had no
> idea of the levels of complexity actually involved!
>
>
> > I can think of a lot of criteria that all-else-being-equal we would
> > like Python to meet. (Of course, in practice they conflict.)
> >
> > Consistency across APIs: it's surprising if locals() and
> > frame.f_locals do different things. This argues for [proxy].
>
> I don't think it is that surprising, since frame.f_locals is kinda
> obscure (the average Python coder wouldn't know a frame if one fell
> on them) and locals() has been documented as weird for decades.
>
> In any case, at least "its a copy of ..." is simple and understandable.
>
>
> > Consistency across contexts: it's surprising if locals() has acts
> > differently in module/class scope versus function scope. This argues
> > for [proxy].
>
> True as far as it goes, but its also true that for the longest time, in
> most implementations, locals() has acted differently. So no change there.
>
> On the other hand, locals() currently returns a dict everywhere. It
> might be surprising for it to start returning a proxy object inside
> functions instead of a dict.
>
>
> --
> Steven
> _______________________________________________
> Python-Dev mailing list
> Python-Dev at python.org
> https://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe:
> https://mail.python.org/mailman/options/python-dev/guido%40python.org
>

-- 
--Guido van Rossum (python.org/~guido)
*Pronouns: he/him/his **(why is my pronoun here?)*
<http://feministing.com/2015/02/03/how-using-they-as-a-singular-pronoun-can-change-the-world/>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20190527/74fc9ef0/attachment.html>