[Markus sent this to me off-list, but agreed to me responding on-list, quoting his entire message.] On Wed, Dec 26, 2012 at 2:38 PM, Markus <nepenthesdev@gmail.com> wrote:

Hi,

Hi Markus, I don't believe we've met before, have we? It would probably help if you introduced yourself and your experience, since our past experiences color our judgment.

as I've been waiting for this to happen, I decided to speak up. While I really look forward to this, I disagree with the PEP.

Heh, we can't all agree on everything. :-)

First shoot should be getting a well established event loop into python.

Perhaps. What is your definition of an event loop?

libev is great, it takes care of operating system specialities, and only does a single job, providing an event loop.

It is also written for C, and I presume much of its API design was influenced by the conventions and affordabilities of that language.

This event loop can take care of timers, sockets and signals,

But sockets are not native on Windows, and I am making some effort with PEP 3156 to efficiently support higher-level abstractions without tying them to sockets. (The plan is to support IOCP on Windows. The previous version of Tulip already had a branch that did support that, as a demonstration of the power of this abstraction.)

pyev, a great python wrapper for libev already provides this simple eventing facility in python.

But, being a libev wrapper, it is likely also strongly influenced by C.

In case you embed python in a c program, the libev default loop of the python code and c code can even be shared, providing a great amount of flexibility.

Only if the C code also uses libev, of course. But C programs may use other event mechanisms -- e.g. AFAIK there are alternatives to libev (during the early stages of Tulip development I chatted a bit with one of the original authors of libevent, Niels Provos, and I believe there's also something called libuv), and GUI frameworks (e.g. X, Qt, Gtk, Wx) tend to have their own event loop. PEP 3156 is designed to let alternative *implementations* of the same *interface* be selected at run time. Hopefully it is possible to provide a conforming implementation using libev -- then your goal (smooth interoperability with C code using libev) is obtained. It's possible that in order to do that the PEP 3156 interface may have to be refactored into separate pieces. The Tulip implementation already has separate "pollster" implementations (which concern themselves *only* with polling for I/O using select, poll, or other alternatives). It probably makes sense to factor the part that implements transports out as well. However, the whole point of including transports and protocols (and futures) in the PEP is that some platforms may want to implement the same high-level API (e.g. create a transport that connects to a certain host/port) using a different approach altogether, e.g. on Windows the transport might not even use sockets. OTOH on UNIX it may be possible to add file descriptors representing pipes and pseudo-ttys.

libev is great as it is small - it provides exactly what's required, and nothing beyond.

Depending on your requirements. :-)

getaddrinfo/getnameinfo/create_transport are out of scope from a event loop point of view. This functionality already exists in python, it just does not use a event loop and is blocking, as every other io related api.

It wasn't random to add these. The "event loop" in PEP 3156 provides abstractions that leave the platform free to implement connections using the appropriate native constructs without letting those constructs "leak" into the application -- after all, whether you're on UNIX or on Windows, a TCP connection represents the same abstraction, but the network stack may have a very different interface.

I'd propose not to replicate the functionality in the event loop namespace, but to extend the existing implementations - by allowing to provide an event loop/callback/ctx as optional args which get used.

That's an interface choice that I would regret (I really don't like writing code using callbacks). (It would also be harder to implement initially as a 3rd party framework. At the lowest level, no changes to Python itself are needed -- it already supports non-blocking sockets, for example. But adding optional callbacks to existing low-level APIs would require changes throughout the stdlib.)

If you specify something like pyev as PEP, you can still come up with another PEP which defines the semantics for upper layer protocols like udp/tcp on IPv4/6, which can be used to take care of dns and 'echo-server-connections'.

I could split up the PEP, but that wouldn't really change anything, since to me it is still a package deal. I am willing to put an effort into specifying a low-level event loop because I know that I can still write high-level code which is (mostly) free of callbacks, using futures, tasks and the yield-from construct. And in order to do that I need a minimum set of high-level abstractions such as getaddrinfo() and transport creation (the exact names of the transport creation methods are still under debate, as are the details of their signatures, but the need for them is established without a doubt in my mind). I note that the stdlib socket module has roughly the same set of abstractions bundled together: - socket objects - getaddrinfo(), getnameinfo() - create_connection() - the makefile() methods on socket objects, which create buffered streams PEP 3156 offers alternatives for all of these, using higher-level abstractions that have been developed and proven in practice by Twisted, *and* offers a path to interop to frameworks that previously couldn't very well interoperate -- Twisted, Tornado, and others have traditionally been pretty segregated, but with PEP 3156 they can interoperate both through the event loop and through Futures (which are friendly both to a callback style and to yield-from).

Anyway, I really hope you'll have a look on libev and pyev, both is great and well tested software and may give you an idea what people who dedicate themselves to event loops came up with already in terms of names, subclassing, requirements, guarantees and workarounds for platform specific failures (kqueue, epoll ...).

I will certainly have a look! I am not so concerned about naming (it seems inevitable that everyone uses somewhat different terminology anyway, and it is probably better not to reuse terms when the meaning is different), but I do like to look at guarantees (or the absence thereof!) and best practices for dealing with the differences between platforms.

http://pod.tst.eu/http://cvs.schmorp.de/libev/ev.pod http://code.google.com/p/pyev/

All together, I'd limit the scope of the PEP to the API of the event loop, just focussing on io/timers/signals and propose to extend existing API to be usable with an event loop, instead of replicating it.

You haven't convinced me about this. However, you can help me by comparing the event loop part of PEP 3156 (ignoring anything that returns or takes a Future) to libev and pointing out things (either specific APIs or certain guarantees or requirements) that would be hard to implement using libev, as well as useful features in libev that you think every event loop should have.

For naming I'd prefer 'watcher' over 'Handler'.

Hm, 'watcher' to me sounds more active than the behavior I have in mind for this class. It is just a reification of a specific function and some arguments to pass to it, with the ability to cancel the call altogether. Thanks for writing! -- --Guido van Rossum (python.org/~guido)