[Python-ideas] asyncore: included batteries don't fit

[Laurens Van Houtven]

Oh my me. This is a very long thread that I probably should have replied to
a long time ago. This thread is intensely long right now, and tonight is
the first chance I've had to try and go through it comprehensively. I'll
try to reply to individual points made in the thread -- if I missed yours,
please don't be offended, I promise it's my fault :)

FYI, I'm the sucker who originally got tricked into starting PEP 3153, aka
async-pep.

First of all, I'm glad to see that there's some more "let's get that pep
along" movement. I tabled it because:

a) I didn't have enough time to contribute,
b) a lot of promised contributions ended up not happening when it came down
to it, which was incredibly demotivating. The combination of this thread,
plus the fact that I was strong armed at Pycon ZA by a bunch of community
members that shall not be named (Alex, Armin, Maciej, Larry ;-)) into
exploring this thing again.

First of all, I don't feel async-pep is an attempt at twisted light in the
stdlib. Other than separation of transport and protocol, there's not really
much there that even smells of twisted (especially since right now I'd
probably throw consumers/producers out) -- and that separation is simply
good practice. Twisted does the same thing, but it didn't invent it.
Furthermore, the advantages seem clear: reusability and testability are
more than enough for me.

If there's one take away idea from async-pep, it's reusable protocols.

The PEP should probably be a number of PEPs. At first sight, it seems that
this number is at least four:

1. Protocol and transport abstractions, making no mention of asynchronous
IO (this is what I want 3153 to be, because it's small, manageable, and
virtually everyone appears to agree it's a fantastic idea)
2. A base reactor interface
3. A way of structuring callbacks: probably deferreds with a built-in
inlineCallbacks for people who want to write synchronous-looking code with
explicit yields for asynchronous procedures
4+ adapting the stdlib tools to using these new things

Re: forward path for existing asyncore code. I don't remember this being
raised as an issue. If anything, it was mentioned in passing, and I think
the answer to it was something to the tune of "asyncore's API is broken,
fixing it is more important than backwards compat". Essentially I agree
with Guido that the important part is an upgrade path to a good third-party
library, which is the part about asyncore that REALLY sucks right now.
Regardless, an API upgrade is probably a good idea. I'm not sure if it
should go in the first PEP: given the separation I've outlined above (which
may be too spread out...), there's no obvious place to put it besides it
being a new PEP.

Re base reactor interface: drawing maximally from the lessons learned in
twisted, I think IReactorCore (start, stop, etc), IReactorTime (call later,
etc), asynchronous-looking name lookup, fd handling are the important
parts. call_every can be implemented in terms of call_later on a separate
object, so I think it should be (eg twisted.internet.task.LoopingCall). One
thing that is apparently forgotten about is event loop integration. The
prime way of having two event loops cooperate is *NOT* "run both in
parallel", it's "have one call the other". Even though not all loops
support this, I think it's important to get this as part of the interface
(raise an exception for all I care if it doesn't work).

cheers
lvh
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20121009/1127e269/attachment.html>