[Twisted-Python] Asynchronous code PEP
Hi! I've just been in contact with Jim Fulton, who was previously mentioned in the context of developing this PEP, and he's okay with us taking over. IIUC the PEP involves a few things: 1. A sane reactor abstraction/interface (although they probably won't like pulling in z.i.I) in the standard library which everything should be able to implement -- essentially twisted.loop's interface. This probably includes a Protocol abstraction too 2. A simple, potentially insane implementation for it in the stdlib -- ideally twisted.loop, but probably something like asyncore-except-slightly-less-terrible 3. A way to write code for it, probably involving @inlineCallbacks/monocle style yields The idea here, as Glyph told me at Pycon, I believe, is that people can just write code that works on most backends. When they figure out the stdlib thing suck^H^H^H^Hdoesn't satisfy their requirements, we can just say "hey, it's okay, you can just replace the reactor reactor you have now with the twisted one and you will get all sorts of wonderful magic to play with". Am I completely wrong here already or does that sound like a sane problem definition? Either way I'm planning to put the PEP draft up on github (actually, I already have). Whether you like github or not, I think this feature: https://github.com/blog/844-forking-with-the-edit-button is just too amazing for writing a PEP as a community process to let pass. cheers lvh
On May 30, 2011, at 7:25 AM, Laurens Van Houtven wrote:
Hi!
I've just been in contact with Jim Fulton, who was previously mentioned in the context of developing this PEP, and he's okay with us taking over.
IIUC the PEP involves a few things: A sane reactor abstraction/interface (although they probably won't like pulling in z.i.I) in the standard library which everything should be able to implement -- essentially twisted.loop's interface. This probably includes a Protocol abstraction too Yes. Practically speaking these will have to be ABCs to fit into the Python stdlib idiom. A simple, potentially insane implementation for it in the stdlib -- ideally twisted.loop, but probably something like asyncore-except-slightly-less-terrible Yes. A way to write code for it, probably involving @inlineCallbacks/monocle style yields No.
In the interests of keeping the scope here as tight as possible, this is *not* a PEP about Deferreds or any of their equivalents, which are substantially more controversial. It is _just_ about an asynchronous networking API. The most important parts are IProtocol/ITransport/IConsumer/IProducer. In order to do anything with these, of course, you need something somewhat like IReactorTCP/IReactorTime, but I would like to have those specified separately in the PEP, because the main interesting thing is just getting event-driven protocol logic based on a common API. The IReactorTCP analogue would just be a stub to get started, not a complete specification of every connection mode you might want. At the very least, there's SSL. Plus, even in TCP, there's a lot of complexity there which you need to modify if you want to have different connection/listening options (just look at the mess on the IPv6 ticket if you think this stuff is simple). Convenience APIs like coroutine scheduling are definitely out of scope.
The idea here, as Glyph told me at Pycon, I believe, is that people can just write code that works on most backends. When they figure out the stdlib thing suck^H^H^H^Hdoesn't satisfy their requirements, we can just say "hey, it's okay, you can just replace the reactor reactor you have now with the twisted one and you will get all sorts of wonderful magic to play with".
In addition, it would be a way for the stdlib to gradually start evolving towards protocol implementations which parse chunks that are fed to them rather than calling recv() and expecting to block.
Am I completely wrong here already or does that sound like a sane problem definition?
Close enough :).
Either way I'm planning to put the PEP draft up on github (actually, I already have). Whether you like github or not, I think this feature: https://github.com/blog/844-forking-with-the-edit-button is just too amazing for writing a PEP as a community process to let pass.
Cool, glad to see the project is online somewhere, finally :).
Hooray! Kevin Horn On Tue, May 31, 2011 at 4:58 PM, Laurens Van Houtven <_@lvh.cc> wrote:
As promised:
https://github.com/lvh/async-pep
_______________________________________________ Twisted-Python mailing list Twisted-Python@twistedmatrix.com http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python
On May 31, 2011, at 5:58 PM, Laurens Van Houtven wrote:
As promised:
Great to see a public start on this. It should really say something about protocol/transport separation early on, though ;).
It should really say something about protocol/transport separation early on, though ;).
Pff, publishing finished work is for academics. Do you want me to be an academic, Glyph? Do you? I've added the stuff I got from Jim. FlowControl seems to be the equivalent of IPushProducer. Why does Protocol and Transport inherit from it? I don't understand why all Protocols and Transports are inherently pausable/resumable. cheers lvh
On Wed, Jun 1, 2011 at 4:21 PM, Laurens Van Houtven <_@lvh.cc> wrote:
It should really say something about protocol/transport separation early
on, though ;).
Pff, publishing finished work is for academics. Do you want me to be an academic, Glyph? Do you?
I've added the stuff I got from Jim. FlowControl seems to be the equivalent of IPushProducer. Why does Protocol and Transport inherit from it? I don't understand why all Protocols and Transports are inherently pausable/resumable.
cheers lvh
Hmmm...you might think about also allowing bytearrays everywhere it says "data must be a bytestring". Maybe there's some problem with that (I haven't done a lot of Python3-style IO yet), but I don't think there is, and if they are to be allowed, it should be explicitly mentioned, IMO. Kevin Horn
On Wed, Jun 1, 2011 at 11:34 PM, Kevin Horn <kevin.horn@gmail.com> wrote:
Hmmm...you might think about also allowing bytearrays everywhere it says "data must be a bytestring". Maybe there's some problem with that (I haven't done a lot of Python3-style IO yet), but I don't think there is, and if they are to be allowed, it should be explicitly mentioned, IMO.
Yay, you're responsible for ticket #1: https://github.com/lvh/async-pep/issues/1
Kevin Horn
-- cheers lvh
... and closed! See people? Feedback works, now give me more of it please ;-) cheers lvh
On Thu, Jun 02, 2011 at 12:50:33AM +0200, Laurens Van Houtven wrote:
See people? Feedback works, now give me more of it please ;-)
Perhaps I'm looking at the wrong place; the "pep-xxxx.rst" I'm looking at via the web interface contains only an short abstract and a rationale that won't be controversial to anyone subscribed to this list. There doesn't seem to be much to give feedback *on*. I guess that means my feedback is 'please write more text'. :)
On Thu, Jun 2, 2011 at 5:56 AM, Tim Allen <screwtape@froup.com> wrote:
On Thu, Jun 02, 2011 at 12:50:33AM +0200, Laurens Van Houtven wrote:
See people? Feedback works, now give me more of it please ;-)
Perhaps I'm looking at the wrong place; the "pep-xxxx.rst" I'm looking at via the web interface contains only an short abstract and a rationale that won't be controversial to anyone subscribed to this list. There doesn't seem to be much to give feedback *on*.
Definitely true! I was holding a few parallel backchannel diiscussions with people that have more background information, if all you have is the repo there's definitely not a lot to do right now, but I'm working on fixing this :)
I guess that means my feedback is 'please write more text'. :)
Noted and appreciated!
_______________________________________________ Twisted-Python mailing list Twisted-Python@twistedmatrix.com http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python
-- cheers lvh
participants (4)
-
Glyph Lefkowitz
-
Kevin Horn
-
Laurens Van Houtven
-
Tim Allen