[Python-ideas] concurrency-local namespaces

[Guido van Rossum]

I'll have to leave this up to you and Nathaniel and others to ponder, I've
run out of time to think about more issues. Sorry! (I do think that the
conclusion of that patch was to go with current_task() intead.)

On Wed, Apr 22, 2015 at 11:37 AM, Yury Selivanov <yselivanov.ml at gmail.com>
wrote:

> Guido,
>
> On 2015-04-22 1:46 PM, Guido van Rossum wrote:
>
>> Hey Nathaniel, can you bring this up in the PEP 492 thread? It sounds like
>> it would be super great if the async with statement would (optionally?)
>> notify the context manager of switching to a different task and back.
>>
> I'm not sure if it's possible to implement.  PEP492 coroutines
> know nothing about the loop that schedules them.
>
> One way to solve this problem of TLS-like storage for coroutines
> in asyncio is to add a notion of 'context' to the loop.
>
> In this code review: https://codereview.appspot.com/87940044/#ps1
> please take a look at patch set 1, where I add
> 'loop.get_context_id()' method.
>
> I also implemented a PoC of local storage based on that patch
> (I can't remember where is it, but it was the reason of the code
> review :) I'll implement it again if we resurrect this idea).
>
> With context being implemented on the loop level (with context_id
> passed along with callbacks) it allowed to implement local storage
> concepts on a lower level than Task object, solving some of
> the problems Nathaniel raised.
>
> All in all, I don't think that to implement local storage we
> need to do something to `async with`, but rather we should
> implement this feature in asyncio and have the local storage
> object shipped with it.
>
> Thanks,
> Yury
>
>
>
>> On Tue, Apr 21, 2015 at 10:36 PM, Nathaniel Smith <njs at pobox.com> wrote:
>>
>>  On Tue, Apr 21, 2015 at 4:48 PM, Eric Snow <ericsnowcurrently at gmail.com>
>>> wrote:
>>>
>>>> threading.local provides thread-local namespaces.  As far as I can
>>>> tell, there isn't an equivalent for coroutines (asyncio), even though
>>>> I would expect they would provide the same benefits.  I'd like to see
>>>> coroutine-local namespaces added and would be happy to work on the
>>>> problem.  I plan on working on a feature that will rely on applying a
>>>> thread-local context and realized that coroutines would need a similar
>>>> treatment.  Also, there are probably a few spots in the stdlib that
>>>> would benefit (e.g. decimal contexts).
>>>>
>>> Possibly this is off-topic for this thread but -- it's a good point
>>> that, currently, when implementing a global state variable (like the
>>> decimal context), one can bind it to a single thread by storing the
>>> state in TLS, but AFAIK there's no way to do this with coroutines.
>>> It's somewhat unpleasant that
>>>
>>>    with decimal.localcontext() as ctx:
>>>        ctx.prec = 10
>>>        values = (yield from db.fetchstuff())
>>>        morecodehere()
>>>
>>> can cause arbitrary other code to run with the given decimal context
>>> in effect, while also failing to guarantee that morecodehere() will
>>> run with any particular decimal context.
>>>
>>> I hoped for a second that "async with" might help with this, but
>>> checking the PEP it doesn't -- to do that it would need support
>>> context manager callbacks that were called whenever the coroutine got
>>> suspended/resumed while the 'with' block was in effect, but that's not
>>> what it does, it just allows the __enter__ and __exit__ methods
>>> themselves to be coroutines.
>>>
>>> And AFAICT the thread Guido linked
>>> (https://github.com/python/asyncio/issues/165) only considers cases
>>> where an async framework (e.g. a web server framework) is the one who
>>> wants to set global variables that can be accessed from within a set
>>> of coroutines. This works out, b/c when the same code controls both
>>> the spawning of coroutines (by creating a new Task) and setting the
>>> variables, so it can bind them together. But that doesn't seem to help
>>> for something like the decimal module context, or numpy's error
>>> handling context
>>> (http://docs.scipy.org/doc/numpy/reference/generated/numpy.errstate.html
>>> ).
>>>
>>> Obviously one can just document that users should always bracket any
>>> yield/await calls with explicit state setting/restore functions, but
>>> is there any better solution?
>>>
>>> -n
>>>
>>> --
>>> Nathaniel J. Smith -- http://vorpus.org
>>> _______________________________________________
>>> Python-ideas mailing list
>>> Python-ideas at python.org
>>> https://mail.python.org/mailman/listinfo/python-ideas
>>> Code of Conduct: http://python.org/psf/codeofconduct/
>>>
>>>
>>
>>
>>
>> _______________________________________________
>> Python-ideas mailing list
>> Python-ideas at python.org
>> https://mail.python.org/mailman/listinfo/python-ideas
>> Code of Conduct: http://python.org/psf/codeofconduct/
>>
>
> _______________________________________________
> Python-ideas mailing list
> Python-ideas at python.org
> https://mail.python.org/mailman/listinfo/python-ideas
> Code of Conduct: http://python.org/psf/codeofconduct/
>



-- 
--Guido van Rossum (python.org/~guido)
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20150422/f03bf1a0/attachment.html>