[Python-ideas] The async API of the future: Twisted and Deferreds

[Laurens Van Houtven]

Interesting. That's certainly a nice API, but that then again (read_until)
sounds like something I'd implement using dataReceived... You know,
read_until clears the buffer, logs the requested callback. data_received
adds something to the buffer, and checks if it triggered the (one of the?)
registered callbacks.

Of course, I may just be rusted in my ways and trying to implement
everything in terms of things I know (then again, that might be just what's
needed when you're trying to make a useful general API).

I guess it's time for me to go deep-diving into Tornado :)

On Sat, Oct 13, 2012 at 7:27 PM, Ben Darnell <ben at bendarnell.com> wrote:

> On Sat, Oct 13, 2012 at 10:18 AM, Laurens Van Houtven <_ at lvh.cc> wrote:
> > What calls on_headers in this example? Coming from twisted, that seems
> like
> > dataReceived's responsibility, but given your introductory paragraph
> that's
> > not actually what goes on here?
>
> The IOStream does, after send_request calls
> stream.read_until("\r\n\r\n", on_headers).  Inside IOStream, there is
> a _handle_read method that is registered with the IOLoop and fills up
> a buffer.  When the read condition is satisfied the IOStream calls
> back into application code.
>
> -Ben
>
> >
> >
> > On Sat, Oct 13, 2012 at 7:07 PM, Ben Darnell <ben at bendarnell.com> wrote:
> >>
> >> On Fri, Oct 12, 2012 at 11:14 PM, Antoine Pitrou <solipsis at pitrou.net>
> >> wrote:
> >> > On Fri, 12 Oct 2012 15:11:54 -0700
> >> > Guido van Rossum <guido at python.org> wrote:
> >> >>
> >> >> > 2. Method dispatch callbacks:
> >> >> >
> >> >> >     Similar to the above, the reactor or somebody has a handle on
> >> >> > your
> >> >> > object, and calls methods that you've defined when events happen
> >> >> >     e.g. IProtocol's dataReceived method
> >> >>
> >> >> While I'm sure it's expedient and captures certain common patterns
> >> >> well, I like this the least of all -- calling fixed methods on an
> >> >> object sounds like a step back; it smells of the old Java way (before
> >> >> it had some equivalent of anonymous functions), and of asyncore,
> which
> >> >> (nearly) everybody agrees is kind of bad due to its insistence that
> >> >> you subclass its classes. (Notice how subclassing as the prevalent
> >> >> approach to structuring your code has gotten into a lot of discredit
> >> >> since 1996.)
> >> >
> >> > But how would you write a dataReceived equivalent then? Would you have
> >> > a "task" looping on a read() call, e.g.
> >> >
> >> > @task
> >> > def my_protocol_main_loop(conn):
> >> >     while <some_condition>:
> >> >         try:
> >> >             data = yield conn.read(1024)
> >> >         except ConnectionError:
> >> >             conn.close()
> >> >             break
> >> >
> >> > I'm not sure I understand the problem with subclassing. It works fine
> >> > in Twisted. Even in Python 3 we don't shy away from subclassing, for
> >> > example the IO stack is based on subclassing RawIOBase,
> BufferedIOBase,
> >> > etc.
> >>
> >> Subclassing per se isn't a problem, but requiring a single
> >> dataReceived method per class can be awkward.  Many protocols are
> >> effectively state machines, and modeling each state as a function can
> >> be cleaner than a big if/switch block in dataReceived.  For example,
> >> here's a simplistic HTTP client using tornado's IOStream:
> >>
> >>        from tornado import ioloop
> >>         from tornado import iostream
> >>         import socket
> >>
> >>         def send_request():
> >>             stream.write("GET / HTTP/1.0\r\nHost: friendfeed.com
> \r\n\r\n")
> >>             stream.read_until("\r\n\r\n", on_headers)
> >>
> >>         def on_headers(data):
> >>             headers = {}
> >>             for line in data.split("\r\n"):
> >>                parts = line.split(":")
> >>                if len(parts) == 2:
> >>                    headers[parts[0].strip()] = parts[1].strip()
> >>             stream.read_bytes(int(headers["Content-Length"]), on_body)
> >>
> >>         def on_body(data):
> >>             print data
> >>             stream.close()
> >>             ioloop.IOLoop.instance().stop()
> >>
> >>         s = socket.socket(socket.AF_INET, socket.SOCK_STREAM, 0)
> >>         stream = iostream.IOStream(s)
> >>         stream.connect(("friendfeed.com", 80), send_request)
> >>         ioloop.IOLoop.instance().start()
> >>
> >>
> >> Classes allow and encourage broader interfaces, which are sometimes a
> >> good thing, but interact poorly with coroutines.  Both twisted and
> >> tornado use separate callbacks for incoming data and for the
> >> connection being closed, but for coroutines it's probably better to
> >> just treat a closed connection as an error on the read.  Futures (and
> >> yield from) give us a nice way to do that.
> >>
> >> -Ben
> >> _______________________________________________
> >> Python-ideas mailing list
> >> Python-ideas at python.org
> >> http://mail.python.org/mailman/listinfo/python-ideas
> >
> >
> >
> >
> > --
> > cheers
> > lvh
> >
>



-- 
cheers
lvh
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20121013/6baf689a/attachment.html>