[Python-ideas] [Python-Dev] Python needs a standard asynchronous return object

[Guido van Rossum]

Moving to python-ideas.

Have you seen http://www.python.org/dev/peps/pep-3148/ ? That seems
exactly what you want.

--Guido

On Fri, Sep 10, 2010 at 4:00 PM, James Yonan <james at openvpn.net> wrote:
> I'd like to propose that the Python community standardize on a "deferred"
> object for asynchronous return values, modeled after the well-thought-out
> Twisted Deferred class.
>
> With more and more Python libraries implementing asynchronicity (for example
> Futures -- PEP 3148), it's crucial to have a standard deferred object in
> place so that code using a single asynchronous reactor can interoperate with
> different asynchronous libraries.
>
> I think a lot of people don't realize how much cooler and more elegant it is
> to return a deferred object from an asynchronous function rather than using
> a generic callback approach (where you pass a function argument to the
> asynchronous function telling it where to call when the asynchronous
> operation completes).
>
> While asynchronous systems have been shown to have excellent scalability
> properties, the callback-based programming style often used in asynchronous
> programming has been criticized for breaking up the sequential readability
> of program logic.
>
> This problem is elegantly addressed by using Deferred Generators.  Since
> Python 2.5 added enhanced generators (i.e. the capability for "yield" to
> return a value), the infrastructure is now in place to allow an asynchronous
> function to be written in a sequential style, without the use of explicit
> callbacks.
>
> See the following blog article for a nice write-up on the capability:
>
> http://blog.mekk.waw.pl/archives/14-Twisted-inlineCallbacks-and-deferredGenerator.html
>
> Mekk's Twisted Deferred example:
>
> @defer.inlineCallbacks
> def someFunction():
>    a = 1
>    b = yield deferredReturningFunction(a)
>    c = yield anotherDeferredReturningFunction(a, b)
>    defer.returnValue(c)
>
> What's cool about this is that between the two yield statements, the Twisted
> reactor is in control meaning that other pending asynchronous tasks can be
> attended to or the thread's remaining time slice can be yielded to the
> kernel, yet this is all accomplished without the use of multi-threading.
>  Another interesting aspect of this approach is that since it leverages on
> Python's enhanced generators, an exception thrown inside either of the
> deferred-returning functions will be propagated through to someFunction()
> where it can be handled with try/except.
>
> Think about what this means -- this sort of emulates the "stackless" design
> pattern you would expect in Erlang or Stackless Python without leaving
> standard Python.  And it's made possible under the hood by Python Enhanced
> Generators.
>
> Needless to say, it would be great to see this coolness be part of the
> standard Python library, instead of having every Python asynchronous library
> implement its own ad-hoc callback system.
>
> James Yonan
> _______________________________________________
> Python-Dev mailing list
> Python-Dev at python.org
> http://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe:
> http://mail.python.org/mailman/options/python-dev/guido%40python.org
>



-- 
--Guido van Rossum (python.org/~guido)