[Python-ideas] PEP draft: context variables

[Koos Zevenhoven]

On Tue, Sep 5, 2017 at 3:49 AM, Nathaniel Smith <njs at pobox.com> wrote:

> On Mon, Sep 4, 2017 at 2:50 PM, Koos Zevenhoven <k7hoven at gmail.com> wrote:
> > Hi all,
> >
> > as promised, here is a draft PEP for context variable semantics and
> > implementation. Apologies for the slight delay; I had a not-so-minor
> > autosave accident and had to retype the majority of this first draft.
> >
> > During the past years, there has been growing interest in something like
> > task-local storage or async-local storage. This PEP proposes an
> alternative
> > approach to solving the problems that are typically stated as motivation
> for
> > such concepts.
>
> From a quick skim, my impression is:
>
>
​Well, I'm happy to hear that a quick skim can already give you an
impression ;).​ But let's see how correct...



> All the high-level semantics you suggest make sense... in fact, AFAICT
> they're exactly the same semantics we've been using as a litmus test
> for PEP 550.


Well, if "exactly the same semantics" is even nearly true, you are only
testing a small subset of PEP 550 which resembles a subset of this proposal.



> I think PEP 550 is sufficient to allow implementing all
> your proposed APIs (and that if it isn't, that's a bug in PEP 550).
>

That's not true either. The LocalContext-based semantics introduces scope
barriers that affect *all* variables. You might get close by putting just
one variable in a LogicalContext and then nest them, but PEP 550 does not
allow this in all cases. With the addition of PEP 521 and some trickery, it
might.

See also this section in PEP 550, where one of the related issues is
described:

https://www.python.org/dev/peps/pep-0550/#should-yield-
from-leak-context-changes



> OTOH, your proposal doesn't provide any way to implement functions
> like decimal.setcontext or numpy.seterr, except by pushing a new state
> and never popping it, which leaks memory and permanently increases the
> N in the O(N) lookups.
>
>
​Well, there are different approaches for this. Let's take the example of
numpy.

import numpy as np

I believe the relevant functions are

np.seterr -- set a new state (and return the old one)
np.geterr -- get the current state
np.errstate -- gives you a context manager to do handle

(Well, errstate sets more state than np.seterr, but that's irrelevant
here).

First of all, the np.seterr API is something that I want to discourage in
this proposal, because if the state is not reset back to what it was, a
completely different piece of code may be affected.

BUT

To preserve the current semantics of these functions in non-async code, you
could do this:

- numpy reimplements the errstate context manager using contextvars based
on this proposal.

- geterr gets the state using contextvars

- seterr gets the state using contextvars and mutates it the way it wants


(If contextvars is not available, it uses the old way)

Also, the idea is to also provide frameworks the means for implementing
concurrency-local storage, if that is what people really want, although I'm
not sure it is.



> I didn't see any direct comparison with PEP 550 in your text (maybe I
> missed it). Why do you think this approach would be better than what's
> in PEP 550?
>

It was not my intention to leave out the comparison altogether, but I did
avoid the comparisons in some cases in this first draft, because thinking
about PEP 550 concepts while trying to understand this proposal might give
you the wrong idea.

One of the benefits of this proposal is simplicity, and I'm guessing
performance as well, but that would need evidence.

––Koos



-- 
+ Koos Zevenhoven + http://twitter.com/k7hoven +
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20170905/4676c10b/attachment.html>