[Python-ideas] PEP 3156/Tulip: Extensible EventLoop interface

[Ben Darnell]

The event loop interface in PEP 3156 has an extensibility problem.  It
seems likely that it will have a method like listen_udp() by the time it's
done, but suppose that doesn't make it into the first official release.
 Third-party event loop implementations may want to provide UDP support as
an extension, but the most consistent way to provide that extension is by
adding new methods on the event loop object, where various extensions risk
conflicting with each other or with new methods that become standardized
later.

The PEP specifies the add_reader family of methods in part so that core
protocol implementations can be shared across all EventLoop implementations
that support these methods.  However, if those transports are implemented
in a common base class (like Twisted's PosixReactorBase), there is no way
for third-party transports to take advantage of a similar structure.  I'd
like to make it possible for transports to be developed independent of a
particular EventLoop implementation in a way that is consistent with the
way the core transports work.  (This is a bigger concern for tulip than it
is for twisted because twisted can update PosixReactorBase more frequently
than the stdlib can change)

I propose turning the interface around so that transport creation uses
static functions that take the event loop as an argument, with a
double-dispatch mechanism to allow the event loop to provide the actual
implementation when it can:

  def create_connection(protocol_factory, host, port, event_loop=None):
    if event_loop is None:
      event_loop = get_event_loop()
    # Note the use of a fully-qualified name in the registry
    impl = event_loop.get_implementation('tulip.create_connection')
    return impl(protocol_factory, host, port)

New third-party transports could provide fallbacks for event loops that
don't have their own implementations:

  if impl is None:
    # These supports_*() functions are placeholders for a to-be-determined
introspection interface.
    if supports_fd_interface(event_loop):
      return posix_udp_implementation(*args)
    elif supports_iocp_interface(event_loop):
      return iocp_udp_implementation(*args)
    else:
      raise Exception("This transport is not supported on this event loop")

Or they could plug into the event loop's implementation registry:

  LibUVEventLoop.register_implementation('mymodule.listen_udp',
libuv_udp_implementation)

This does introduce a little magic (is there any precedent for this kind of
multiple-dispatch in the standard library?), but I like the way it keeps
the event loop interface from getting too big and monolithic.  Third-party
transports can avoid naming conflicts without looking fundamentally
different from standard ones, and there's a clean path from doing something
that's platform-specific (e.g. with add_reader and friends) to supporting
multiple event loops to full standardization.

-Ben
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20130202/2cf5fcc7/attachment.html>