[Python-ideas] asyncore: included batteries don't fit

[chrysn]

On Mon, Sep 24, 2012 at 03:31:37PM -0700, Giampaolo Rodolà wrote:
> From a chronological standpoint I still think the best thing to do in order
> to fix the "python async problem" once and for all is to first define and
> possibly implement an "async WSGI interface" describing what a standard
> async IO loop/reactor should look like (in terms of API) and how to
> integrate with it, see:
> http://mail.python.org/pipermail/python-ideas/2012-May/015223.html
> http://mail.python.org/pipermail/python-ideas/2012-May/015235.html

i wasn't aware that pep 3153 exists. given that, my original intention
of this thread should be re-worded into "let's get pep3153 along!".

i'm not convinced by the api suggested in the first mail, as it sounds
very unix centric (poll, read/write/error). i rather imagined leaving
the details of the callbackable/mainloop interaction to be platform
details. (a win32evtlog event source just couldn't possibly register
with a select() based main loop). i'd prefer to keep the part that
registers with the a main loop concentrated to a very lowlevel common
denominator. for unix, that'd mean that there is a basic callbackable
for "things that receive events because they have a fileno". everything
above that, eg the distinction whether a "w" event means that we can
write() or that we must accept() could happen above that and wouldn't
have to be concerned with the main loop integration any more.

in case (pseudo)code gets the idea over better:

class UnixFilehandle(object):
    def __init__(self, fileno):
        self._fileno = fileno

    def register_with_main_loop(self, mainloop):
        # it might happen that the main loop doesn't support unix
	# filenos. tough luck, in that case -- the developer should
	# select a more suitable main loop.
        mainloop.register_unix_fileno(self._fileno, self)

    def handle_r_event(self): raise NotImplementedError("Not configured to receive that sort of event")
    # if you're sure you'd never receive any anyway, you can
    # not-register them by setting them None in the subclass
    handle_w_event = handle_e_event = handle_r_event

class SocketServer(UnixFilehandle):
    def __init__(self, socket):
        self._socket = socket
	UnixFilehandle.init(socket.fileno())

    def handle_w_event(self):
        self.handle_accept_event(self.socket.accept())

other interfaces parallel to the file handle interface would, for
example, handle unix signals. (built atop of that, like the
accept-handling socket server, could be an that deals with child
processes.) the interface for android might look different again,
because there is no main loop and select never gets called by the
application.

> From there the python stdlib *might* grow a new module implementing the
> "async WSGI interface" (let's call it asyncore2) and some of the stdlib
> batteries such as socketserver can possibly use it.
> 
> In my mind this is the ideal long-term scenario but even managing to define
> an "async WSGI interface" alone would be a big step forward.

i'd welcome such an interface. if asyncore can then be retrofitted to
accept that interface too w/o breaking compatibility, it'd be nice, but
if not, it's asyncore2, then.

> Again, at this point in time what you're proposing looks too vague,
> ambitious and premature to me.

please don't get me wrong -- i'm not proposing anything for immediate
action, i just want to start a thinking process towards a better
integrated stdlib.


On Mon, Sep 24, 2012 at 05:02:08PM -0700, Josiah Carlson wrote:
> 1. Whatever reactors are available, you need to be able to instantiate
> multiple of different types of reactors and multiple instances of the
> same type of reactor simultaneously (to support multiple threads
> handling different groups of reactors, or different reactors for
> different types of objects on certain platforms). While this allows
> for insanity in the worst-case, we're all consenting adults here, so
> shouldn't be limited by reactor singletons. There should be a default
> reactor class, which is defined on module/package import (use the
> "best" one for the platform).

i think that's already common. with asyncore, you can have different
maps (just one is installed globally as default). with the gtk main
loop, it's a little tricky (the gtk.main() function doesn't simply take
an argument), but the underlying glib can do that afaict.

> 2. The API must be simple. I am not sure that it can get easier than
> Idea #3 from:
> http://mail.python.org/pipermail/python-ideas/2012-May/015245.html

it's good that the necessities of call_later and call_every are
mentioned here, i'd have forgotten about them.


we've talked about many things we'd need in a python asynchronous
interface (not implementation), so what are the things we *don't* need?
(so we won't start building a framework like twisted). i'll start:

* high-level protocol handling (can be extra modules atop of it)
* ssl
* something like the twisted delayed framework (not sure about that, i
  guess the twisted people will have good reason to use it, but i don't
  see compelling reasons for such a thing in a minimal interface from my
  limited pov)
* explicit connection handling (retries, timeouts -- would be up to the
  user as well, eg urllib might want to set up a timeout and retries for
  asynchronous url requests)

best regards
chrysn

-- 
To use raw power is to make yourself infinitely vulnerable to greater powers.
  -- Bene Gesserit axiom
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 836 bytes
Desc: Digital signature
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20120926/44c24a49/attachment.pgp>