[Python-ideas] concurrency-local namespaces
yselivanov.ml at gmail.com
Wed Apr 22 20:51:27 CEST 2015
On 2015-04-22 2:45 PM, Guido van Rossum wrote:
> I'll have to leave this up to you and Nathaniel and others to ponder, I've
> run out of time to think about more issues. Sorry! (I do think that the
> conclusion of that patch was to go with current_task() intead.)
Yes, that was the conclusion. Because I was interested to add
this functionality for web frameworks, where, as Nathaniel
mentioned, you control the top-level Task object. So I gave up
on pushing the 'context' idea.
If we want to add a generic thread-local-like object to
coroutines, I think that it has to be done on an asyncio
event loop level. Because event loop is not just about
coroutines, it's also about callbacks, and it is useful
to *avoid* loosing context in them.
Nathaniel, I'd be glad if you could glance over my old proposal.
It's based on ideas I expressed in this thread:
> On Wed, Apr 22, 2015 at 11:37 AM, Yury Selivanov <yselivanov.ml at gmail.com>
>> On 2015-04-22 1:46 PM, Guido van Rossum wrote:
>>> Hey Nathaniel, can you bring this up in the PEP 492 thread? It sounds like
>>> it would be super great if the async with statement would (optionally?)
>>> notify the context manager of switching to a different task and back.
>> I'm not sure if it's possible to implement. PEP492 coroutines
>> know nothing about the loop that schedules them.
>> One way to solve this problem of TLS-like storage for coroutines
>> in asyncio is to add a notion of 'context' to the loop.
>> In this code review: https://codereview.appspot.com/87940044/#ps1
>> please take a look at patch set 1, where I add
>> 'loop.get_context_id()' method.
>> I also implemented a PoC of local storage based on that patch
>> (I can't remember where is it, but it was the reason of the code
>> review :) I'll implement it again if we resurrect this idea).
>> With context being implemented on the loop level (with context_id
>> passed along with callbacks) it allowed to implement local storage
>> concepts on a lower level than Task object, solving some of
>> the problems Nathaniel raised.
>> All in all, I don't think that to implement local storage we
>> need to do something to `async with`, but rather we should
>> implement this feature in asyncio and have the local storage
>> object shipped with it.
>>> On Tue, Apr 21, 2015 at 10:36 PM, Nathaniel Smith <njs at pobox.com> wrote:
>>> On Tue, Apr 21, 2015 at 4:48 PM, Eric Snow <ericsnowcurrently at gmail.com>
>>>>> threading.local provides thread-local namespaces. As far as I can
>>>>> tell, there isn't an equivalent for coroutines (asyncio), even though
>>>>> I would expect they would provide the same benefits. I'd like to see
>>>>> coroutine-local namespaces added and would be happy to work on the
>>>>> problem. I plan on working on a feature that will rely on applying a
>>>>> thread-local context and realized that coroutines would need a similar
>>>>> treatment. Also, there are probably a few spots in the stdlib that
>>>>> would benefit (e.g. decimal contexts).
>>>> Possibly this is off-topic for this thread but -- it's a good point
>>>> that, currently, when implementing a global state variable (like the
>>>> decimal context), one can bind it to a single thread by storing the
>>>> state in TLS, but AFAIK there's no way to do this with coroutines.
>>>> It's somewhat unpleasant that
>>>> with decimal.localcontext() as ctx:
>>>> ctx.prec = 10
>>>> values = (yield from db.fetchstuff())
>>>> can cause arbitrary other code to run with the given decimal context
>>>> in effect, while also failing to guarantee that morecodehere() will
>>>> run with any particular decimal context.
>>>> I hoped for a second that "async with" might help with this, but
>>>> checking the PEP it doesn't -- to do that it would need support
>>>> context manager callbacks that were called whenever the coroutine got
>>>> suspended/resumed while the 'with' block was in effect, but that's not
>>>> what it does, it just allows the __enter__ and __exit__ methods
>>>> themselves to be coroutines.
>>>> And AFAICT the thread Guido linked
>>>> (https://github.com/python/asyncio/issues/165) only considers cases
>>>> where an async framework (e.g. a web server framework) is the one who
>>>> wants to set global variables that can be accessed from within a set
>>>> of coroutines. This works out, b/c when the same code controls both
>>>> the spawning of coroutines (by creating a new Task) and setting the
>>>> variables, so it can bind them together. But that doesn't seem to help
>>>> for something like the decimal module context, or numpy's error
>>>> handling context
>>>> Obviously one can just document that users should always bracket any
>>>> yield/await calls with explicit state setting/restore functions, but
>>>> is there any better solution?
>>>> Nathaniel J. Smith -- http://vorpus.org
>>>> Python-ideas mailing list
>>>> Python-ideas at python.org
>>>> Code of Conduct: http://python.org/psf/codeofconduct/
>>> Python-ideas mailing list
>>> Python-ideas at python.org
>>> Code of Conduct: http://python.org/psf/codeofconduct/
>> Python-ideas mailing list
>> Python-ideas at python.org
>> Code of Conduct: http://python.org/psf/codeofconduct/
More information about the Python-ideas