On Sat, Oct 13, 2012 at 12:13 PM, Laurens Van Houtven <_@lvh.cc> wrote:
I quite like IOStream's interface, actually. If that's part of the transport layer, how do you prevent from having duplicating its behavior (read_until etc)? If there's just another separate object that would be the ITransport in twisted, I think the difference is purely one of labeling.
So far we haven't actually needed much flexibility in the transport layer - most of the functionality is in the BaseIOStream class, and then there are subclasses IOStream (regular sockets), SSLIOStream, and PipeIOStream that actually call recv(), read(), connect(), etc. We might need a little refactoring if we introduce dramatically different types of transports, but the plan is that we'd represent transports as classes in the IOStream hierarchy. -Ben
On Sat, Oct 13, 2012 at 8:54 PM, Ben Darnell
wrote: On Sat, Oct 13, 2012 at 10:49 AM, Laurens Van Houtven <_@lvh.cc> wrote:
Interesting. That's certainly a nice API, but that then again (read_until) sounds like something I'd implement using dataReceived... You know, read_until clears the buffer, logs the requested callback. data_received adds something to the buffer, and checks if it triggered the (one of the?) registered callbacks.
Right, that's how IOStream is implemented internally. The transport/protocol split works a little differently in Tornado: IOStream is implemented something like a Protocol subclass, but we consider it a part of the transport layer. The "protocols" are arbitrary classes that don't share any particular interface, but instead just call methods on the IOStream.
-Ben
Of course, I may just be rusted in my ways and trying to implement everything in terms of things I know (then again, that might be just what's needed when you're trying to make a useful general API).
I guess it's time for me to go deep-diving into Tornado :)
On Sat, Oct 13, 2012 at 7:27 PM, Ben Darnell
wrote: On Sat, Oct 13, 2012 at 10:18 AM, Laurens Van Houtven <_@lvh.cc> wrote:
What calls on_headers in this example? Coming from twisted, that seems like dataReceived's responsibility, but given your introductory paragraph that's not actually what goes on here?
The IOStream does, after send_request calls stream.read_until("\r\n\r\n", on_headers). Inside IOStream, there is a _handle_read method that is registered with the IOLoop and fills up a buffer. When the read condition is satisfied the IOStream calls back into application code.
-Ben
On Sat, Oct 13, 2012 at 7:07 PM, Ben Darnell
wrote: On Fri, Oct 12, 2012 at 11:14 PM, Antoine Pitrou
wrote: > On Fri, 12 Oct 2012 15:11:54 -0700 > Guido van Rossum wrote: >> >> > 2. Method dispatch callbacks: >> > >> > Similar to the above, the reactor or somebody has a handle >> > on >> > your >> > object, and calls methods that you've defined when events >> > happen >> > e.g. IProtocol's dataReceived method >> >> While I'm sure it's expedient and captures certain common >> patterns >> well, I like this the least of all -- calling fixed methods on an >> object sounds like a step back; it smells of the old Java way >> (before >> it had some equivalent of anonymous functions), and of asyncore, >> which >> (nearly) everybody agrees is kind of bad due to its insistence >> that >> you subclass its classes. (Notice how subclassing as the >> prevalent >> approach to structuring your code has gotten into a lot of >> discredit >> since 1996.) > > But how would you write a dataReceived equivalent then? Would you > have > a "task" looping on a read() call, e.g. > > @task > def my_protocol_main_loop(conn): > while : > try: > data = yield conn.read(1024) > except ConnectionError: > conn.close() > break > > I'm not sure I understand the problem with subclassing. It works > fine > in Twisted. Even in Python 3 we don't shy away from subclassing, > for > example the IO stack is based on subclassing RawIOBase, > BufferedIOBase, > etc. Subclassing per se isn't a problem, but requiring a single dataReceived method per class can be awkward. Many protocols are effectively state machines, and modeling each state as a function can be cleaner than a big if/switch block in dataReceived. For example, here's a simplistic HTTP client using tornado's IOStream:
from tornado import ioloop from tornado import iostream import socket
def send_request(): stream.write("GET / HTTP/1.0\r\nHost: friendfeed.com\r\n\r\n") stream.read_until("\r\n\r\n", on_headers)
def on_headers(data): headers = {} for line in data.split("\r\n"): parts = line.split(":") if len(parts) == 2: headers[parts[0].strip()] = parts[1].strip() stream.read_bytes(int(headers["Content-Length"]), on_body)
def on_body(data): print data stream.close() ioloop.IOLoop.instance().stop()
s = socket.socket(socket.AF_INET, socket.SOCK_STREAM, 0) stream = iostream.IOStream(s) stream.connect(("friendfeed.com", 80), send_request) ioloop.IOLoop.instance().start()
Classes allow and encourage broader interfaces, which are sometimes a good thing, but interact poorly with coroutines. Both twisted and tornado use separate callbacks for incoming data and for the connection being closed, but for coroutines it's probably better to just treat a closed connection as an error on the read. Futures (and yield from) give us a nice way to do that.
-Ben _______________________________________________ Python-ideas mailing list Python-ideas@python.org http://mail.python.org/mailman/listinfo/python-ideas
-- cheers lvh
-- cheers lvh
-- cheers lvh