[Python-ideas] The async API of the future: Twisted and Deferreds

[Ben Darnell]

On Sat, Oct 13, 2012 at 10:18 AM, Laurens Van Houtven <_ at lvh.cc> wrote:
> What calls on_headers in this example? Coming from twisted, that seems like
> dataReceived's responsibility, but given your introductory paragraph that's
> not actually what goes on here?

The IOStream does, after send_request calls
stream.read_until("\r\n\r\n", on_headers).  Inside IOStream, there is
a _handle_read method that is registered with the IOLoop and fills up
a buffer.  When the read condition is satisfied the IOStream calls
back into application code.

-Ben

>
>
> On Sat, Oct 13, 2012 at 7:07 PM, Ben Darnell <ben at bendarnell.com> wrote:
>>
>> On Fri, Oct 12, 2012 at 11:14 PM, Antoine Pitrou <solipsis at pitrou.net>
>> wrote:
>> > On Fri, 12 Oct 2012 15:11:54 -0700
>> > Guido van Rossum <guido at python.org> wrote:
>> >>
>> >> > 2. Method dispatch callbacks:
>> >> >
>> >> >     Similar to the above, the reactor or somebody has a handle on
>> >> > your
>> >> > object, and calls methods that you've defined when events happen
>> >> >     e.g. IProtocol's dataReceived method
>> >>
>> >> While I'm sure it's expedient and captures certain common patterns
>> >> well, I like this the least of all -- calling fixed methods on an
>> >> object sounds like a step back; it smells of the old Java way (before
>> >> it had some equivalent of anonymous functions), and of asyncore, which
>> >> (nearly) everybody agrees is kind of bad due to its insistence that
>> >> you subclass its classes. (Notice how subclassing as the prevalent
>> >> approach to structuring your code has gotten into a lot of discredit
>> >> since 1996.)
>> >
>> > But how would you write a dataReceived equivalent then? Would you have
>> > a "task" looping on a read() call, e.g.
>> >
>> > @task
>> > def my_protocol_main_loop(conn):
>> >     while <some_condition>:
>> >         try:
>> >             data = yield conn.read(1024)
>> >         except ConnectionError:
>> >             conn.close()
>> >             break
>> >
>> > I'm not sure I understand the problem with subclassing. It works fine
>> > in Twisted. Even in Python 3 we don't shy away from subclassing, for
>> > example the IO stack is based on subclassing RawIOBase, BufferedIOBase,
>> > etc.
>>
>> Subclassing per se isn't a problem, but requiring a single
>> dataReceived method per class can be awkward.  Many protocols are
>> effectively state machines, and modeling each state as a function can
>> be cleaner than a big if/switch block in dataReceived.  For example,
>> here's a simplistic HTTP client using tornado's IOStream:
>>
>>        from tornado import ioloop
>>         from tornado import iostream
>>         import socket
>>
>>         def send_request():
>>             stream.write("GET / HTTP/1.0\r\nHost: friendfeed.com\r\n\r\n")
>>             stream.read_until("\r\n\r\n", on_headers)
>>
>>         def on_headers(data):
>>             headers = {}
>>             for line in data.split("\r\n"):
>>                parts = line.split(":")
>>                if len(parts) == 2:
>>                    headers[parts[0].strip()] = parts[1].strip()
>>             stream.read_bytes(int(headers["Content-Length"]), on_body)
>>
>>         def on_body(data):
>>             print data
>>             stream.close()
>>             ioloop.IOLoop.instance().stop()
>>
>>         s = socket.socket(socket.AF_INET, socket.SOCK_STREAM, 0)
>>         stream = iostream.IOStream(s)
>>         stream.connect(("friendfeed.com", 80), send_request)
>>         ioloop.IOLoop.instance().start()
>>
>>
>> Classes allow and encourage broader interfaces, which are sometimes a
>> good thing, but interact poorly with coroutines.  Both twisted and
>> tornado use separate callbacks for incoming data and for the
>> connection being closed, but for coroutines it's probably better to
>> just treat a closed connection as an error on the read.  Futures (and
>> yield from) give us a nice way to do that.
>>
>> -Ben
>> _______________________________________________
>> Python-ideas mailing list
>> Python-ideas at python.org
>> http://mail.python.org/mailman/listinfo/python-ideas
>
>
>
>
> --
> cheers
> lvh
>