[Python-ideas] The async API of the future

[Greg Ewing]

Guido van Rossum wrote:
> I would like people to be able to write fast
> event handling programs on Windows too, ... But I don't know how
> tenable that is given the dramatically different style used by IOCP
> and the need to use native Windows API for all async I/O -- it sounds
> like we could only do this if the library providing the I/O loop
> implementation also wrapped all I/O operations, and that may be a bit
> much.

That's been bothering me, too. It seems like an interface
accommodating the completion-based style will have to be
*extremely* fat.

That's not just a burden for anyone implementing the
interface, it's a problem for any library wanting to *wrap*
it as well.

For example, to maintain separation between the async
layer and the generator layer, we will probably want to
have an AsyncSocket object in the async layer, and a
separate GeneratorSocket in the generator layer that wraps
an AsyncSocket.

If the AsyncSocket needs to provide methods for all the
possible I/O operations that one might want to perform on
a socket, then GeneratorSocket needs to provide its own
versions of all those methods as well.

Multiply that by the number of different kinds of I/O
objects (files, sockets, message queues, etc. -- there
seem to be quite a lot of them on Windows) and that's
a *lot* of stuff to be wrapped.

> Finally, there should also be some minimal interface so that multiple
> I/O loops can interact -- at least in the case where one I/O loop
> belongs to a GUI library.

That's another thing that worries me. With a ready-based
event loop, this is fairly straightforward. If you can get
hold of the file descriptor or handle that the GUI is
ultimately reading its input from, all you need to do is
add it as an event source to your main loop, and when it's
ready, tell the GUI event loop to run itself once.

But you can't do that with a completion-based main loop,
because the actual reading of the input needs to be done
in a different way, and that's usually buried somewhere
deep in the GUI library where you can't easily change it.

 > It seems this is a solved problem (as well
> solved as you can hope for) to Twisted, so we should just adopt their
> approach.

Do they actually do it for an IOCP-based main loop on Windows?
If so, I'd be interested to know how.

-- 
Greg