[Python-ideas] Async context managers and iterators with tulip

[Guido van Rossum]

On Sat, Dec 22, 2012 at 1:14 AM, Nick Coghlan <ncoghlan at gmail.com> wrote:
> On Sat, Dec 22, 2012 at 4:17 PM, guido.van.rossum
> <python-checkins at python.org> wrote:
>> +- We might introduce explicit locks, though these will be a bit of a
>> +  pain to use, as we can't use the ``with lock: block`` syntax
>> +  (because to wait for a lock we'd have to use ``yield from``, which
>> +  the ``with`` statement can't do).
>
> Actually, I just realised that the following can work if the async
> lock is defined appropriately:
>
>     with yield from async_lock:
>         ...

Syntactically you'd have to say

    with (yield from async_lock):
        ....

> The secret is that async_lock would need to be a coroutine rather than
> a context manager. *Calling* the coroutine would acquire the lock
> (potentially registering a callback that is scheduled when the lock is
> released) and return a context manager that released the lock. The
> async_lock itself wouldn't be a context manager, so you'd get an
> immediate error if you left out the "yield from".

Very nice.

> We'd be heading even further down the path of
> two-languages-for-the-price-of-one if we did that, though (by which I
> mean the fact that async code and synchronous code exist in parallel
> universes - one, more familiar one, where the ability to block is
> assumed, as is the fact that any operation may give concurrent code
> the chance to execute, and the universe of Twisted, tulip, et al,
> where possible suspension points are required to be explicitly marked
> in the function where they occur).

It's inevitable that some patterns work well together while others
don't. I see no big philosophical problem with this. Pragmatically,
we'll have plenty of places where existing stdlib modules can't be
used with tulip, and the tulip-compatible upgrade will have a
different API. (The trickiest part will be that the classic code, e.g.
urllib, must work in any thread and cannot rely on the existence of an
event loop. *Maybe* you can get by with
get_event_loop().run_until_complete(<future>) but that might still
depend on the default event loop policy. Food for thought.)

-- 
--Guido van Rossum (python.org/~guido)