[Python-Dev] Timeout for PEP 550 / Execution Context discussion
yselivanov.ml at gmail.com
Mon Oct 16 12:11:26 EDT 2017
On Mon, Oct 16, 2017 at 11:49 AM, Guido van Rossum <guido at python.org> wrote:
> On Sun, Oct 15, 2017 at 10:26 PM, Nathaniel Smith <njs at pobox.com> wrote:
>> On Sun, Oct 15, 2017 at 10:10 PM, Guido van Rossum <guido at python.org>
>> > Yes, that's what I meant by "ignoring generators". And I'd like there to
>> > be
>> > a "current context" that's a per-thread MutableMapping with ContextVar
>> > keys.
>> > Maybe there's not much more to it apart from naming the APIs for getting
>> > and
>> > setting it? To be clear, I am fine with this being a specific subtype of
>> > MutableMapping. But I don't see much benefit in making it more abstract
>> > than
>> > that.
>> We don't need it to be abstract (it's fine to have a single concrete
>> mapping type that we always use internally), but I think we do want it
>> to be opaque (instead of exposing the MutableMapping interface, the
>> only way to get/set specific values should be through the ContextVar
>> interface). The advantages are:
>> - This allows C level caching of values in ContextVar objects (in
>> particular, funneling mutations through a limited API makes cache
>> invalidation *much* easier)
> Well the MutableMapping could still be a proxy or something that invalidates
> the cache when mutated. That's why I said it should be a single concrete
> mapping type. (It also doesn't have to derive from MutableMapping -- it's
> sufficient for it to be a duck type for one, or perhaps some Python-level
> code could `register()` it.
Yeah, we can do a proxy.
>> - It gives us flexibility to change the underlying data structure
>> without breaking API, or for different implementations to make
>> different choices -- in particular, it's not clear whether a dict or
>> HAMT is better, and it's not clear whether a regular dict or
>> WeakKeyDict is better.
> I would keep it simple and supid, but WeakKeyDict is a subtype of
> MutableMapping, and I'm sure we can find a way to implement the full
> MutableMapping interface on top of HAMT as well.
>> The first point (caching) I think is the really compelling one: in
>> practice decimal and numpy are already using tricky caching code to
>> reduce the overhead of accessing the ThreadState dict, and this gets
>> even trickier with context-local state which has more cache
>> invalidation points, so if we don't do this in the interpreter then it
>> could actually become a blocker for adoption. OTOH it's easy for the
>> interpreter itself to do this caching, and it makes everyone faster.
> I agree, but I don't see how making the type a subtype (or duck type) of
> MutableMapping prevents any of those strategies. (Maybe you were equating
> MutableMapping with "subtype of dict"?)
Question: why do we want EC objects to be mappings? I'd rather make
them opaque, which will result in less code and make it more
The key arguments for keeping ContextVar abstraction:
* Naturally avoids name clashes.
* Allows to implement efficient caching. This is important if we want
libraries like decimal/numpy to start using it.
* Abstracts away the actual implementation of the EC. This is a
future-proof solution, with which we can enable EC support for
generators in the future. We already know two possible solutions (PEP
550 v1, PEP 550 current), and ContextVar is a good enough abstraction
to support both of them.
IMO ContextVar.set() and ContextVar.get() is a simple and nice API to
work with the EC. Most people (aside framework authors) won't even
need to work with EC objects directly anyways.
More information about the Python-Dev