On Wed, Apr 29, 2015 at 3:01 PM, Yury Selivanov <yselivanov.ml@gmail.com> wrote:
Hi Nathanial,
Sorry for not replying to your last email promptly. FWIW I was going to do it later today ;)
My opinion on this subject (and I've implemented lots of local-context kind of objects in different frameworks) is that *just* inserting some kind of suspend/resume points before/after yields does not work.
Good point.
Why:
1. Greenlets, gevent, eventlet, stackless python: they do not have 'yield'. Context switches are invisible to the interpreter. And those frameworks are popular. You want numpy.errstate/decimal.localcontext to work there too.
2. I'm curious how this will be implemented and what's the performance impact.
3. I think that mechanism should be more generic than just 'with' staments. What if you want to have a decorator that applies some context? What if you can't write it as a generator with @contextlib.contextmanager?
What I would propose:
I think that the right approach here would be to have a standard protocol; something similar to how we defined WSGI -- it doesn't require any changes in the interpreter, it is a protocol.
For instance, we could have a module in the stdlib, with a class Context:
class Context: @classmethod def __suspend_contexts(cls): for child in cls.__subclasses__(): child.__suspend_context()
@classmethod def __resume_contexts(cls): ..
To inform context objects that the context is about to change, asyncio event loop would call Context.__suspend_contexts()
I think that it's up to the event loop/library/framework to manage context switches properly. In asyncio, for instance, you can attach callbacks to Futures. I think it would be great if such callbacks are executed in the right context.
I would also suggest to think about a universal context -- the one that works both for asyncio coroutines and threadpools they might use.
This way any framework can implement the context in the most efficient way.
All in all, I'm in favor of API and a couple functions added to the stdlib for this.
Great idea. I've been thinking of something along those lines. I think there also needs to be a recognition of active vs. inactive context. Only active contexts would be used by the event loop/etc. A generic context would also make it easy to clone and to use in context managers. One approach: class Context: def activate(self): ... # or open def deactivate(self): ... # or close def clone(self): ... def __suspend__(self, cs, id): ... def __resume__(self, cs, id): ... def __enter__(self): self.activate() return self def __exit__(self, ...): self.deactivate() def active(ctx=Context): return ... The various "context switchers" (event loop, thread pool, etc.) would then call __suspend__ and __resume__ on all the active contexts. The two args (cs for "context switcher") to those methods would allow each context object to adjust to the targeted "linear execution context" uniquely identified by the (cs, id) pair. For example: thread/async-local namespaces. I've often found it to be cleaner to split the "spec" from the "instance", so: class InactiveContext: # or GlobalContext or DefaultContext def activate(self): ... # return a new cloned Context def clone(self): ... # merge with activate? class Context: def deactivate(self): ... def clone(self): ... def __suspend__(self, cm, id): ... def __resume__(self, cm, id): ... def __enter__(self): return self def __exit__(self, ...): self.deactivate() This is all rough and the result of divided attention over the last few hours, but the idea of leaving the switching up to the "context switcher" is the key thing. Also, I smell some overlapping concepts with existing types and patterns that could probably distill the idea of a generic Context type into something cleaner. -eric