[Python-ideas] Add a generic async IO poller/reactor to select module

[Giampaolo Rodolà]

2012/5/24 Nick Coghlan <ncoghlan at gmail.com>:
> On Thu, May 24, 2012 at 10:37 PM, Nick Coghlan <ncoghlan at gmail.com> wrote:
>> On Thu, May 24, 2012 at 9:50 PM, Giampaolo Rodolà <g.rodola at gmail.com> wrote:
>>> poller.poll serves the same purpose of asyncore.loop, yes, but this is
>>> supposed to be independent from asyncore.
>>
>> I'd actually like to see something like this pitched as a
>> "concurrent.eventloop" PEP. PEP 3153 really wasn't what I was
>> expecting after the discussions at the PyCon US 2011 language summit -
>> I was expecting "here's a common event loop all the async frameworks
>> can hook into", but instead we got something a *lot* more ambitious
>> taht tried to merge the entire IO stack for the async frameworks,
>> rather than just provide a standard way for their event loops to
>> cooperate.
>
> See the final section of my notes here:
> http://www.boredomandlaziness.org/2011/03/python-language-summit-rough-notes.html
>
> Turns out the idea of a PEP 3153 level API *was* raised at the summit,
> but I'd still like to see a competing PEP that targets the reactor
> level API directly.
>
> Cheers,
> Nick.


It's not clear to me what such a PEP should address in particular,
anyway here's a bunch of semi-random ideas.


=== Idea #1 ===

4 classes (SelectPoller, PollPoller, EpollPoller, KqueuePoller) within
concurrent.eventloop namespace all sharing the same API:

- register(fd, events, callback)  # callback gets called with events as arg
- modify(fd, events)
- unregister(fd)
- call_later(timeout, callback, errback=None)
- call_every(timeout, callback, errback=None)
- poll(timeout=1.0, blocking=True)
- close()

call_later() and call_every() can return an object having cancel() and
reset() methods.
The user willing to register a new handler will do:

>>> poller.register(sock.fileno(), poller.READ | poller.WRITE, callback)

...then, in the callback:

def callback(events):
    if events & poller.ERROR and not events & poller.READ:
          disconnect()
    else:
         if events & poller.READ:
             read()
         if events & poller.WRITE:
             write()


pros: highly customizable
cons: too low level, requires manual handling

=== Idea #2 ===

same as #1 except:

- register(fd, events)
- poll(timeout=1.0)  # desn't block, return {fd:events, fd:events, ...}


=== Idea #3 ===

same as #1 except:

- register(fd, events, handler)
- poll(timeout=1.0, blocking=True)

...poll() will call handler.handle_X_event() depending on the current
event (READ, WRITE or ERROR).
An internal map such as {fd:handler, fd:handler} will be maintaned internally.

- pros: easier to use
- cons: more rigid, requires a "contract" with the handler


--- Giampaolo
http://code.google.com/p/pyftpdlib/
http://code.google.com/p/psutil/
http://code.google.com/p/pysendfile/