[Python-ideas] PEP 3156 feedback

[Giampaolo Rodolà]

2012/12/18 Guido van Rossum <guido at python.org>
>
> On Tue, Dec 18, 2012 at 2:01 AM, Antoine Pitrou <solipsis at pitrou.net> wrote:> Event loop API
> > --------------
> >
> > I would like to say that I prefer Tornado's model: for each primitive
> > provided by Tornado, you can pass an explicit Loop instance which you
> > instantiated manually.
> > There is no module function or policy object hiding this mechanism:
> > it's simple, explicit and flexible (in other words: if you want a
> > per-thread event loop, just do it yourself using TLS :-)).
>
> It sounds though as if the explicit loop is optional, and still
> defaults to some global default loop?
>
> Having one global loop shared by multiple threads is iffy though. Only
> one thread should be *running* the loop, otherwise the loop can' be
> used as a mutual exclusion device. Worse, all primitives for adding
> and removing callbacks/handlers must be made threadsafe, and then
> basically the entire event loop becomes full of locks, which seems
> wrong to me.

The basic idea is to have multiple threads/processes, each running its
own IO loop.
No locks are required because each IO poller instance will deal with
its own socket-map / callbacks-queue and no resources are shared.
In asyncore this was achieved by introducing the "map" parameter.
Similarly to Tornado, pyftpdlib uses an "ioloop" parameter which can
be passed to all the classes which will handle the connection (the
handlers).
If "ioloop" is provided all the handlers will use that (...and
register() against it, add_reader() etc..) otherwise the "global"
ioloop instance will be used (default).
A dynamic IO poller like this is important because in case the
connection handlers are forced to block for some reason, you can
switch from a concurrency model (async / non-blocking) to another
(multi threads/process) very easily.
See:
http://code.google.com/p/pyftpdlib/issues/detail?id=212#c9
http://code.google.com/p/pyftpdlib/source/browse/trunk/pyftpdlib/servers.py?spec=svn1137&r=1137

Hope this helps,

--- Giampaolo
http://code.google.com/p/pyftpdlib/
http://code.google.com/p/psutil/
http://code.google.com/p/pysendfile/