[Python-ideas] Async API: some code to review

Guido van Rossum guido at python.org
Mon Oct 29 00:52:02 CET 2012

I am finally ready to show the code I worked on for the past two
weeks. This is definitely not ready for anything except as a quick
demo, but I learned enough while writing it to feel comfortable with
the PEP 380 paradigm.

I've set up a Hg repo on code.google.com, and I picked a codename:
tulip. View the code here:

It runs on Linux and OSX; I have no easy access to Windows but I'd be
happy to take contributions.

Key files in the directory:

- main.py: the main program for testing, and a rough HTTP client
- sockets.py: transports for sockets and SSL, and a buffering layer
- scheduling.py: a Task class and related stuff; this is where the PEP
380 scheduler is implemented
- polling.py: an event loop and basic polling implementations for:
select(), poll(), epoll(), kqueue()

Other junk: .hgignore, Makefile, README, p3time.py (benchmark yield
from vs. plain functions), longlines.py (stupid style checker)

More detailed discussions per file follows; please read the code along
with my description (separately they may not make much sense):

polling.py: http://code.google.com/p/tulip/source/browse/polling.py

I found it remarkably easy to come up with polling implementations
using all those different system calls. I ended up mixing in the
pollster class with the event loop class, although I'm not sure that's
the best design -- perhaps it's better if the event loop just
references the pollster as a separate object.

The pollster has a very simple API: add_reader(fd, callback, *args),
add_writer(<ditto>), remove_reader(fd), remove_writer(fd), and
poll(timeout) -> list of events. (fd means file descriptor.) There's
also pollable() which just checks if there are any fds registered. My
implementation requires fd to be an int, but that could easily be
extended to support other types of event sources. I'm not super happy
that I have parallel reader/writer APIs, but passing a separate
read/write flag didn't come out any more elegant, and I don't foresee
other operation types (though I may be wrong).

The event list started out as a tuple of (fd, flag, callback, args),
where flag is 'r' or 'w' (easily extensible); in practice neither the
fd nor the flag are used, and one of the last things I did was to wrap
callback and args into a simple object that allows cancelling the
callback; the add_*() methods return this object. (This could probably
use a little more abstraction.) Note that poll() doesn't call the
callbacks -- that's up to the event loop.

The event loop has two basic ways to register callbacks:
call_soon(callback, *args) causes callback(*args) to be called the
next time the event loop runs; call_later(delay, callback, *args)
schedules a callback at some time (relative or absolute) in the
future. It also inherits add_reader() and add_writer() from the
pollster. Then there is run(), which runs the event loop until there's
nothing left to do (no readers, no writers, no soon or later
callbacks), and run_once(), which goes through the entire list of
event sources once. (I think the order in which I do this isn't quite
right but it works for now.)

Finally, there's a helper class (ThreadRunner) here which lets you run
something in a separate thread using the features of
concurrent.futures. It uses the "self-pipe trick" (Google it :-) to
ensure that the poll() call wakes up -- this is needed by
call_in_thread() at the next layer (scheduling.py). (There may be a
race condition here, but I think it can be fixed.)

Note that there are no yields (or yield froms) here; that's for the next layer:

scheduling.py: http://code.google.com/p/tulip/source/browse/scheduling.py

This is the scheduler for PEP-380 style coroutines. I started with a
Scheduler class and operations along the lines of Greg Ewing's design,
with a Scheduler instance as a global variable, but ended up ripping
it out in favor of a Task object that represents a single stack of
generators chained via yield-from. There is a Context object holding
the event loop and the current task in thread-local storage, so that
multiple threads can (and must) have independent event loops.

Most user (and much library) code in this system should be written as
generators invoking other generators directly using yield from.
However to run something as an independent task, you wrap the
generator call in a Task() constructor, possibly giving it a timeout,
and then calling its start() method. A Task also acts a little like a
future -- you can wait() for it, add done-callbacks, and it preserves
the return value of the generator call. This can be used to introduce
concurrency or to give something a separate timeout. (There are also
primitives to wait for the first N completed of a bunch of Tasks.)

To invoke a primitive I/O operation, you call the current task's
block() method and then immediately yield (similar to Greg Ewing's
approach). There are helpers block_r() and block_w() that arrange for
a task to block until a file descriptor is ready for reading/writing.
Examples of their use are in sockets.py.

There is also call_in_thread() which integrates with
polling.ThreadRunner to run a function in a separate thread and wait
for it. Also used in sockets.py.

In the docstrings I use the prefix "COROUTINE:" to indicate public
APIs that should be invoked using yield from.

sockets.py: http://code.google.com/p/tulip/source/browse/sockets.py

This implements some internet primitives using the APIs in
scheduling.py (including block_r() and block_w()). I call them
transports but they are different from transports Twisted; they are
closer to idealized sockets. SocketTransport wraps a plain socket,
offering recv() and send() methods that must be invoked using yield
from. SslTransport wraps an ssl socket (luckily in Python 2.6 and up,
stdlib ssl sockets have good async support!). Then there is a
BufferedReader class that implements more traditional read() and
readline() coroutines (i.e., to be invoked using yield from), the
latter handy for line-oriented transports. Finally there are some
functions for connecting sockets, the highest-level one
create_transport(). These use call_in_thread() to run
socket.getaddrinfo() in a thread (this provides IPv6 support).

I don't particularly care about the exact abstractions in this module;
they are convenient and I was surprised how easy it was to add SSL,
but still these mostly serve as somewhat realistic examples of how to
use scheduling.py. (Afterthought: I think the SocketTransport's recv()
and send() methods could be made more similar to SslTransport.)

More examples in the final file:

main.py: http://code.google.com/p/tulip/source/browse/main.py

There is a simplistic HTTP client here built on top of the
sockets.*Transport abstractions. And the main code exercises this by
spawning four tasks fetching a variety of URLs (more when you
uncomment a block of code) and waiting for their results. The code is
a bit of a mess because I used it as a place to try out various APIs.

I'm most interested in feedback on the design of polling.py and
scheduling.py, and to a lesser extent on the design of sockets.py;
main.py is just an example of how this style works out in practice.

Sorry for the brain-dump style; I would like to write it all up
better, but at the same time waiting longer doesn't necessarily make
it better, so here it is, for all to see. (I also have a list of
problems I had to debug during the development and what I learned from
that; but that's too raw to post right now.)

--Guido van Rossum (python.org/~guido)

More information about the Python-ideas mailing list