[Python-ideas] Async API

[Sam Rushing]

On 10/23/12 12:33 PM, Yury Selivanov wrote:
> The whole point of this text was to show, that pure 'yield' approach will not
> work.  Moreover, I don't think it's time to pronounce "The Right Way" of 'yielding'
> and 'yield-fromming'.  There are so many ways of doing that: with @coroutine
> decorator, plain generators, futures and Tasks, and perhaps more.  And I honestly
> don't know with one is the right one.

[Thanks Yury for giving me a convenient place to jump in]

I abandoned the callback-driven approach in 1999, after pushing it as
far as I could handle.  IMHO you can build single pieces in a relatively
clean fashion, but you cannot easily combine those pieces together to
build real systems.

Over the past year I've played a little with some generator-based code
(tlslite & bluelets for example), and I don't think they're much of an
improvement.  Whether it's decorated callbacks, generators, whatever, it
all reminds me of impenetrable monad code in Haskell. 
Continuation-passing-style isn't something that humans should be
expected to do, it's a trick for compilers. 8^)

> What we really need now (and I think Guido has already mentioned that) is a 
> callback-based (Deferreds, Futures, plain callbacks) design that is easy to 
> plug-and-play in any coroutine-framework.  It has to be low-level and simple.  
> Sort of WSGI for async frameworks ;)

I've been trying to play catch-up since being told about this thread a
couple of days ago.  If I understand it correctly, 'yield-from' looks
like it can help make generator-based-concurrency a little more sane by
cutting back on endless chains of 'for x in ...: yield ...', right? 
That certainly sounds like an improvement, but does the generator nature
of the API bubble all the way up to the top?  Can you send an email with
a function call?

> We also need to work on the stdlib, so that it is easy to inject a custom socket 
> in any object.  Ideally, by passing it in the constructor (as everybody hates 
> monkey-patching.)
>

I second this one.  Having a way to [optionally] pass in a factory for
sockets would help with portability, and would cut down on the
temptation to monkey-patch.

It'd be really great to use standard 'async' protocol implementations in
a performant way... although I'm not sure how/if I can wedge such code
into systems like shrapnel*, but it all starts with being able to pass
in a socket-like object (or factory).

-Sam

(*) Since no one else has mentioned it yet, a tiny plug here for
shrapnel: https://github.com/ironport/shrapnel


-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 194 bytes
Desc: OpenPGP digital signature
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20121023/37400eb4/attachment.pgp>