[Python-ideas] async: feedback on EventLoop API

[Geert Jansen]

Hi,

below is some feedback on the EventLoop API as implemented in tulip.
I am interested in this for an (alternate) dbus interface that I've written
for Python that supports evented IO. I'm hoping tulip's EventLoop could be an
abstraction as well as a default implementation that allows me to support
just one event interface.

I looked at it from two angles:

 1. Does EventLoop provide everything that is needed from a library writer
    point of view?
 2. Can EventLoop efficiently expose a subset of the functionality of
    some of the main event loop implementations out there today
    (i looked at libuv, libev and Qt).

First some code pointers...

 * https://github.com/geertj/looping - Here i've implemented the EventLoop
   interface for libuv, libev and Qt. It includes a slightly modified version of
   tulip's "polling.py" where I've implemented some of the suggestions below.
   It also adds support for Python 2.6/2.7 as the Python Qt interface (PySide)
   doesn't support Python 3 yet.

 * https://github.com/geertj/python-dbusx - A Python interface for libdbus that
   supports evented IO using an EventLoop interface. This module is also
   tests all the different loops from "looping" by doing D-BUS tests with them
   (looping itself doesn't have tests yet).

My main points of feedback are below:

* It would be nice to have repeatable timers. Repeatable timers are expected
  for example by libdbus when integrating it with an event loop.

  Without repeatable timers, I could emulate a repeatable timer by using
  call_later() and adding a new timer every time the timer fires. This would
  be an inefficient interface though for event loops that natively support
  repeatable timers.

  This could possibly be done by adding a "repeat" argument to call_later().

* It would be nice to be a way to call a callback once per loop iteration.
  An example here is dispatching in libdbus. The easiest way to do this is
  to call dbus_connection_dispatch() every iteration of the loop (a more
  complicated way exists to get notifications when the dispatch status
  changes, but it is edge triggered and difficult to get right).

  This could possibly be implemented by adding a "repeat" argument to
  call_soon().

* A useful semantic for run_once() would be to run the callbacks for
  readers and writers in the same iteration as when the FD got ready.

  This allows for the idiom below when expecting a single event to happen
  on a file descriptor from outside the event loop:

    # handle_read() sets the "ready" flag
    loop.add_reader(fd, handle_read)
    while not ready:
        loop.run_once()

  I use this idiom for example in a blocking method_call() method that calls
  into a D-BUS method.

  Currently, the handle_read() callback would be called in the iteration
  *after* the FD became readable. So this would not work, unless some more
  IO becomes available.

  As far as I can see libev, libuv and Qt all work like this.

* If remove_reader() / remove_writer() would accept the DelayedCall instance
  returned by their add_xxx() cousins, then that would allow for multiple
  callbacks per FD. Not all event loops support this (libuv doesn't, libev
  and Qt do), but for the ones that do could have their functionality could
  be exposed like this. For event loops that don't support this, an exception
  could be raised when adding multiple callbacks per FD.

  Support for multiple callbacks per FD could be advertised as a capability.

* After a DelayedCall is cancelled, it would also be very useful to have a
  second method to enable it again. Having that functionality is more
  efficient than creating a new event. For example, the D-BUS event loop
  integration API has specific methods for toggling events on and off that
  you need to provide.

* (Nitpick) Multiplexing absolute and relative timeouts for the "when"
  argument in call_later() is a little too smart in my view and can lead
  to bugs.

With some input, I'd be happy to produce patches.

Regards,
Geert Jansen