Moving to python-ideas. Have you seen http://www.python.org/dev/peps/pep-3148/ ? That seems exactly what you want. --Guido On Fri, Sep 10, 2010 at 4:00 PM, James Yonan <james@openvpn.net> wrote:

I'd like to propose that the Python community standardize on a "deferred" object for asynchronous return values, modeled after the well-thought-out Twisted Deferred class.

With more and more Python libraries implementing asynchronicity (for example Futures -- PEP 3148), it's crucial to have a standard deferred object in place so that code using a single asynchronous reactor can interoperate with different asynchronous libraries.

I think a lot of people don't realize how much cooler and more elegant it is to return a deferred object from an asynchronous function rather than using a generic callback approach (where you pass a function argument to the asynchronous function telling it where to call when the asynchronous operation completes).

While asynchronous systems have been shown to have excellent scalability properties, the callback-based programming style often used in asynchronous programming has been criticized for breaking up the sequential readability of program logic.

This problem is elegantly addressed by using Deferred Generators.  Since Python 2.5 added enhanced generators (i.e. the capability for "yield" to return a value), the infrastructure is now in place to allow an asynchronous function to be written in a sequential style, without the use of explicit callbacks.

See the following blog article for a nice write-up on the capability:

http://blog.mekk.waw.pl/archives/14-Twisted-inlineCallbacks-and-deferredGene...

Mekk's Twisted Deferred example:

@defer.inlineCallbacks def someFunction():    a = 1    b = yield deferredReturningFunction(a)    c = yield anotherDeferredReturningFunction(a, b)    defer.returnValue(c)

What's cool about this is that between the two yield statements, the Twisted reactor is in control meaning that other pending asynchronous tasks can be attended to or the thread's remaining time slice can be yielded to the kernel, yet this is all accomplished without the use of multi-threading.  Another interesting aspect of this approach is that since it leverages on Python's enhanced generators, an exception thrown inside either of the deferred-returning functions will be propagated through to someFunction() where it can be handled with try/except.

Think about what this means -- this sort of emulates the "stackless" design pattern you would expect in Erlang or Stackless Python without leaving standard Python.  And it's made possible under the hood by Python Enhanced Generators.

Needless to say, it would be great to see this coolness be part of the standard Python library, instead of having every Python asynchronous library implement its own ad-hoc callback system.

James Yonan _______________________________________________ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/guido%40python.org

-- --Guido van Rossum (python.org/~guido)

James Yonan <james <at> openvpn.net> writes:

I'd like to propose that the Python community standardize on a "deferred" object for asynchronous return values, modeled after the well-thought-out Twisted Deferred class.

Ummm, sorry, I seem to need new glasses :-( ignore my last post.

Moving to python-ideas. Have you seen http://www.python.org/dev/peps/pep-3148/ ? That seems exactly what you want. --Guido On Fri, Sep 10, 2010 at 4:00 PM, James Yonan <james@openvpn.net> wrote:

I'd like to propose that the Python community standardize on a "deferred" object for asynchronous return values, modeled after the well-thought-out Twisted Deferred class.

With more and more Python libraries implementing asynchronicity (for example Futures -- PEP 3148), it's crucial to have a standard deferred object in place so that code using a single asynchronous reactor can interoperate with different asynchronous libraries.

I think a lot of people don't realize how much cooler and more elegant it is to return a deferred object from an asynchronous function rather than using a generic callback approach (where you pass a function argument to the asynchronous function telling it where to call when the asynchronous operation completes).

While asynchronous systems have been shown to have excellent scalability properties, the callback-based programming style often used in asynchronous programming has been criticized for breaking up the sequential readability of program logic.

This problem is elegantly addressed by using Deferred Generators.  Since Python 2.5 added enhanced generators (i.e. the capability for "yield" to return a value), the infrastructure is now in place to allow an asynchronous function to be written in a sequential style, without the use of explicit callbacks.

See the following blog article for a nice write-up on the capability:

http://blog.mekk.waw.pl/archives/14-Twisted-inlineCallbacks-and-deferredGene...

Mekk's Twisted Deferred example:

@defer.inlineCallbacks def someFunction():    a = 1    b = yield deferredReturningFunction(a)    c = yield anotherDeferredReturningFunction(a, b)    defer.returnValue(c)

What's cool about this is that between the two yield statements, the Twisted reactor is in control meaning that other pending asynchronous tasks can be attended to or the thread's remaining time slice can be yielded to the kernel, yet this is all accomplished without the use of multi-threading.  Another interesting aspect of this approach is that since it leverages on Python's enhanced generators, an exception thrown inside either of the deferred-returning functions will be propagated through to someFunction() where it can be handled with try/except.

Think about what this means -- this sort of emulates the "stackless" design pattern you would expect in Erlang or Stackless Python without leaving standard Python.  And it's made possible under the hood by Python Enhanced Generators.

Needless to say, it would be great to see this coolness be part of the standard Python library, instead of having every Python asynchronous library implement its own ad-hoc callback system.

James Yonan _______________________________________________ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/guido%40python.org

-- --Guido van Rossum (python.org/~guido)

James Yonan <james <at> openvpn.net> writes:

I'd like to propose that the Python community standardize on a "deferred" object for asynchronous return values, modeled after the well-thought-out Twisted Deferred class.

Ummm, sorry, I seem to need new glasses :-( ignore my last post.