[Python-ideas] The async API of the future: yield-from

[Greg Ewing]

Piet Delport wrote:

> In particular, this suspend() could be used to integrate fairly directly
> with callback-based APIs: for example, if you have a Twisted Deferred,
> you could do:
> 
>     result = yield suspend(d.addCallback)

I've been thinking about how to express this using the
primitives provided by the scheduler in my tutorial.
I don't actually have a primitive that simply suspends
a task; instead, I have one that moves the current task
from the ready list to a specified list:

    scheduler.block(queue)

Similarly, I don't have a primitive that explicitly
resumes a particular task[1] -- only one that takes the first
task off a specified list and resumes it:

    scheduler.unblock(queue)

I think this is a good idea if we want to be able to
cancel tasks, because a cancelled task ought to cleanly
disappear from the system, without any risk that something
will try to schedule it again. This is achievable if we
maintain the invariant that a task always belongs to some
queue, and the scheduler knows about that queue.

Given these primitives, we can define

    def wakeup_callback(queue):
       lambda: scheduler.unblock(queue)

and

    def wait_for_callback(add_callback):
       q = []
       add_callback(wakeup_callback(q))
       scheduler.block(q)
       yield

This is starting to look rather like a semaphore. If we
assume semaphores as a facility provided by the library,
then it becomes very straightforward:

    def wait_for_callback(add_callback):
       s = Semaphore()
       add_callback(s.notify)
       yield from s.wait()

That assumes the callback is single-use. But a semaphore
can also handle multi-use callbacks: you just keep the
semaphore around and repeatedly wait on it. You will get
woken up once for each time the callback is called.

    s = Semaphore()
    something.add_callback(s.notify)
    while we_are_still_interested():
       yield from s.wait()
       ...

---

[1] Actually I do, but I'm thinking it shouldn't be
exposed as part of the public API for reasons given here.

-- 
Greg