[Python-ideas] The async API of the future: Reactors

[Antoine Pitrou]

Hello Guido,

On Fri, 12 Oct 2012 11:13:23 -0700
Guido van Rossum <guido at python.org> wrote:
> OTOH someone else might prefer a buffered stream
> abstraction that just keeps filling its read buffer (and draining its
> write buffer) using level-triggered callbacks, at least up to a
> certain buffer size -- we have to be robust here and make it
> impossible for an evil client to fill up all our memory without our
> approval!

I'd like to know what a sane buffered API for non-blocking I/O may look
like, because right now it doesn't seem to make a lot of sense. At
least this bug is tricky to resolve:
http://bugs.python.org/issue13322

> - There's an abstract Reactor class and an abstract Async I/O object
> class. To get a reactor to call you back, you must give it an I/O
> object, a callback, and maybe some more stuff. (I have gone back and
> like passing optional args for the callback, rather than requiring
> lambdas to create closures.) Note that the callback is *not* a
> designated method on the I/O object!

Why isn't it? In practice, you need several callbacks: in Twisted
parlance, you have dataReceived but also e.g. ConnectionLost
(depending on the transport, you may even imagine other callbacks, for
example for things happening on the TLS layer?).

> - In systems supporting file descriptors, there's a reactor
> implementation that knows how to use select/poll/etc., and there are
> concrete I/O object classes that wrap file descriptors. On Windows,
> those would only be socket file descriptors. On Unix, any file
> descriptor would do.

Windows *is* able to do async I/O on things other than sockets (see the
discussion about IOCP). It's just that the Windows implementation of
select() (the POSIX function call) is limited to sockets.

Regards

Antoine.


-- 
Software development and contracting: http://pro.pitrou.net