[Python-ideas] The async API of the future: yield-from

Greg Ewing greg.ewing at canterbury.ac.nz
Wed Oct 17 23:49:48 CEST 2012

Piet Delport wrote:

> In particular, this suspend() could be used to integrate fairly directly
> with callback-based APIs: for example, if you have a Twisted Deferred,
> you could do:
>     result = yield suspend(d.addCallback)

I've been thinking about how to express this using the
primitives provided by the scheduler in my tutorial.
I don't actually have a primitive that simply suspends
a task; instead, I have one that moves the current task
from the ready list to a specified list:


Similarly, I don't have a primitive that explicitly
resumes a particular task[1] -- only one that takes the first
task off a specified list and resumes it:


I think this is a good idea if we want to be able to
cancel tasks, because a cancelled task ought to cleanly
disappear from the system, without any risk that something
will try to schedule it again. This is achievable if we
maintain the invariant that a task always belongs to some
queue, and the scheduler knows about that queue.

Given these primitives, we can define

    def wakeup_callback(queue):
       lambda: scheduler.unblock(queue)


    def wait_for_callback(add_callback):
       q = []

This is starting to look rather like a semaphore. If we
assume semaphores as a facility provided by the library,
then it becomes very straightforward:

    def wait_for_callback(add_callback):
       s = Semaphore()
       yield from s.wait()

That assumes the callback is single-use. But a semaphore
can also handle multi-use callbacks: you just keep the
semaphore around and repeatedly wait on it. You will get
woken up once for each time the callback is called.

    s = Semaphore()
    while we_are_still_interested():
       yield from s.wait()


[1] Actually I do, but I'm thinking it shouldn't be
exposed as part of the public API for reasons given here.


More information about the Python-ideas mailing list