[Python-ideas] concurrency-local namespaces

[Yury Selivanov]

Guido,

On 2015-04-22 1:46 PM, Guido van Rossum wrote:
> Hey Nathaniel, can you bring this up in the PEP 492 thread? It sounds like
> it would be super great if the async with statement would (optionally?)
> notify the context manager of switching to a different task and back.
I'm not sure if it's possible to implement.  PEP492 coroutines
know nothing about the loop that schedules them.

One way to solve this problem of TLS-like storage for coroutines
in asyncio is to add a notion of 'context' to the loop.

In this code review: https://codereview.appspot.com/87940044/#ps1
please take a look at patch set 1, where I add
'loop.get_context_id()' method.

I also implemented a PoC of local storage based on that patch
(I can't remember where is it, but it was the reason of the code
review :) I'll implement it again if we resurrect this idea).

With context being implemented on the loop level (with context_id
passed along with callbacks) it allowed to implement local storage
concepts on a lower level than Task object, solving some of
the problems Nathaniel raised.

All in all, I don't think that to implement local storage we
need to do something to `async with`, but rather we should
implement this feature in asyncio and have the local storage
object shipped with it.

Thanks,
Yury

>
> On Tue, Apr 21, 2015 at 10:36 PM, Nathaniel Smith <njs at pobox.com> wrote:
>
>> On Tue, Apr 21, 2015 at 4:48 PM, Eric Snow <ericsnowcurrently at gmail.com>
>> wrote:
>>> threading.local provides thread-local namespaces.  As far as I can
>>> tell, there isn't an equivalent for coroutines (asyncio), even though
>>> I would expect they would provide the same benefits.  I'd like to see
>>> coroutine-local namespaces added and would be happy to work on the
>>> problem.  I plan on working on a feature that will rely on applying a
>>> thread-local context and realized that coroutines would need a similar
>>> treatment.  Also, there are probably a few spots in the stdlib that
>>> would benefit (e.g. decimal contexts).
>> Possibly this is off-topic for this thread but -- it's a good point
>> that, currently, when implementing a global state variable (like the
>> decimal context), one can bind it to a single thread by storing the
>> state in TLS, but AFAIK there's no way to do this with coroutines.
>> It's somewhat unpleasant that
>>
>>    with decimal.localcontext() as ctx:
>>        ctx.prec = 10
>>        values = (yield from db.fetchstuff())
>>        morecodehere()
>>
>> can cause arbitrary other code to run with the given decimal context
>> in effect, while also failing to guarantee that morecodehere() will
>> run with any particular decimal context.
>>
>> I hoped for a second that "async with" might help with this, but
>> checking the PEP it doesn't -- to do that it would need support
>> context manager callbacks that were called whenever the coroutine got
>> suspended/resumed while the 'with' block was in effect, but that's not
>> what it does, it just allows the __enter__ and __exit__ methods
>> themselves to be coroutines.
>>
>> And AFAICT the thread Guido linked
>> (https://github.com/python/asyncio/issues/165) only considers cases
>> where an async framework (e.g. a web server framework) is the one who
>> wants to set global variables that can be accessed from within a set
>> of coroutines. This works out, b/c when the same code controls both
>> the spawning of coroutines (by creating a new Task) and setting the
>> variables, so it can bind them together. But that doesn't seem to help
>> for something like the decimal module context, or numpy's error
>> handling context
>> (http://docs.scipy.org/doc/numpy/reference/generated/numpy.errstate.html).
>>
>> Obviously one can just document that users should always bracket any
>> yield/await calls with explicit state setting/restore functions, but
>> is there any better solution?
>>
>> -n
>>
>> --
>> Nathaniel J. Smith -- http://vorpus.org
>> _______________________________________________
>> Python-ideas mailing list
>> Python-ideas at python.org
>> https://mail.python.org/mailman/listinfo/python-ideas
>> Code of Conduct: http://python.org/psf/codeofconduct/
>>
>
>
>
>
> _______________________________________________
> Python-ideas mailing list
> Python-ideas at python.org
> https://mail.python.org/mailman/listinfo/python-ideas
> Code of Conduct: http://python.org/psf/codeofconduct/