[Python-ideas] Async API: some code to review

[Laurens Van Houtven]

Hi,

I've been following the PEP380-related threads and I've reviewed this
stuff, while trying to do the protocols/transports PEP, and trying to glue
the two together.

The biggest difference I can see is that protocols as they've been
discussed are "pull": they get called when some data arrives. They don't
know how much data there is; they just get told "here's some data". The
obvious difference with the API in, eg:

https://code.google.com/p/tulip/source/browse/sockets.py#56

... is that now I have to tell a socket to read n bytes, which "blocks" the
coroutine, then I get some data.

Now, there doesn't have to be an issue; you could simply say:

data = yield from s.recv(4096) # that's the magic number usually right
proto.data_received(4096)

It seems a bit boilerplatey, but I suppose that eventually could be hidden
away.

But this style is pervasive, for example that's how reading by lines works:

https://code.google.com/p/tulip/source/browse/echosvr.py#20

While I'm not a big fan (I may be convinced if I see a protocol test that
looks nice); I'm just wondering if there's any point in trying to write the
pull-style protocols when this works quite differently.

Additionally, I'm not sure if readline belongs on the socket. I understand
the simile with files, though. With the coroutine style I could see how the
most obvious fit would be something like tornado's read_until, or an
as_lines that essentially calls read_until repeatedly. Can the delimiter
for this be modified?

My main syntactic gripe is that when I write @inlineCallbacks code or
monocle code or whatever, when I say "yield" I'm yielding to the reactor.
That makes sense to me (I realize natural language arguments don't always
make sense in a programming language context). "yield from" less so (but
okay, that's what it has to look like). But this just seems weird to me:

yield from trans.send(line.upper())

Not only do I not understand why I'm yielding there in the first place (I
don't have to wait for anything, I just want to push some data out!), it
feels like all of my yields have been replaced with yield froms for no
obvious reason (well, there are reasons, I'm just trying to look at this
naively).

I guess Twisted gets away with this because of deferred chaining: that one
deferred might have tons of callbacks in the background, many of which also
doing IO operations, resulting in a sequence of asynchronous operations
that only at the end cause the generator to be run some more.

I guess that belongs in a different thread, though. Even, then, I'm not
sure if I'm uncomfortable because I'm seeing something different from what
I'm used to, or if my argument from English actually makes any sense
whatsoever.

Speaking of protocol tests, what would those look like? How do I yell, say,
"POST /blah HTTP/1.1\r\n" from a transport? Presumably I'd have a mock
transport, and call the handler with that? (I realize it's early days to be
thinking that far ahead; I'm just trying to figure out how I can contribute
a good protocol definition to all of this).

cheers
lvh
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20121030/3936ab1a/attachment.html>