Temporarily un-lurking to reply to this thread (which I'll actually be reading). Giampaolo and I talked about this for a bit over the weekend, and I have to say that I agree with his perspective. In particular, to get something better than asyncore, there must be something minimally better to build upon. I don't really have an opinion on what that minimally better thing should be named, but I do agree that having a simple reactor API that has predictable behavior over the variety of handlers (select, poll, epoll, kqueue, WSAEvent in Windows, etc.) is necessary. Now, let's get to brass tacks... 1. Whatever reactors are available, you need to be able to instantiate multiple of different types of reactors and multiple instances of the same type of reactor simultaneously (to support multiple threads handling different groups of reactors, or different reactors for different types of objects on certain platforms). While this allows for insanity in the worst-case, we're all consenting adults here, so shouldn't be limited by reactor singletons. There should be a default reactor class, which is defined on module/package import (use the "best" one for the platform). 2. The API must be simple. I am not sure that it can get easier than Idea #3 from: http://mail.python.org/pipermail/python-ideas/2012-May/015245.html I personally like it because it offers a simple upgrade path for asyncore users (create your asyncore-derived classes, pass it into the new reactor), while simultaneously defining a relatively easy API for any 3rd party to integrate with. By offering an easy-to-integrate method for 3rd parties (that is also sane), there is the added bonus that 3rd parties are more likely to integrate, rather than replace, which means more use in the "real world", better bug reports, etc. To simplify integration further, make the API register(fd, handler, events=singleton). Passing no events from the caller means "register me for all events", which will help 3rd parties that aren't great with handling read/write registration. 3. I don't have a 3rd tack, you can hang things on the wall with 2 ;) Regards, - Josiah On Mon, Sep 24, 2012 at 3:31 PM, Giampaolo Rodolà <g.rodola@gmail.com> wrote:
I still think this proposal is too vaguely defined and any effort towards adding async IO support to existing batteries is premature for different reasons, first of which the inadequacy of asyncore as the base async framework to fulfill the task you're proposing.
asyncore is so old and difficult to fix/enhance without breaking backward compatibility (see for example http://bugs.python.org/issue11273#msg156439) that relying on it for any modern work is inevitably a bad idea.
From a chronological standpoint I still think the best thing to do in order to fix the "python async problem" once and for all is to first define and possibly implement an "async WSGI interface" describing what a standard async IO loop/reactor should look like (in terms of API) and how to integrate with it, see: http://mail.python.org/pipermail/python-ideas/2012-May/015223.html http://mail.python.org/pipermail/python-ideas/2012-May/015235.html
From there the python stdlib *might* grow a new module implementing the "async WSGI interface" (let's call it asyncore2) and some of the stdlib batteries such as socketserver can possibly use it.
In my mind this is the ideal long-term scenario but even managing to define an "async WSGI interface" alone would be a big step forward.
Again, at this point in time what you're proposing looks too vague, ambitious and premature to me.
--- Giampaolo http://code.google.com/p/pyftpdlib/ http://code.google.com/p/psutil/ http://code.google.com/p/pysendfile/
2012/9/22 chrysn <chrysn@fsfe.org>
hello python-ideas,
i'd like to start discussion about the state of asyncore/asynchat's adaption in the python standard library, with the intention of finding a roadmap for how to improve things, and of kicking off and coordinating implementations.
here's the problem (as previously described in [issue15978] and redirected here, with some additions):
the asyncore module would be much more useful if it were well integrated in the standard library. in particular, it should be supported by:
* subprocess
* BaseHTTPServer / http.server (and thus, socketserver)
* urllib2 / urllib, http.client
* probably many other network libraries except smtpd, which already uses asyncore
* third party libraries (if stdlib leads the way, the ecosystem will follow; eg pyserial)
without widespread asyncore support, it is not possible to easily integrate different servers and services with each other; with asyncore support, it's just a matter of creating the objects and entering the main loop. (eg, a http server for controlling a serial device, with a telnet-like debugging interface).