[Python-Dev] [PEP 3148] futures - execute computations asynchronously

[Phillip J. Eby]

At 01:03 AM 3/5/2010, Brian Quinlan wrote:
>Hi all,
>
>I recently submitted a daft PEP for a package designed to make it
>easier to execute Python functions asynchronously using threads and
>processes. It lets the user focus on their computational problem
>without having to build explicit thread/process pools and work queues.
>
>The package has been discussed on stdlib-sig but now I'd like this
>group's feedback.

My immediate reaction is that this would be a lot more useful if it 
built on an API for coroutine yielding/interaction, similar to what's 
in say, Eventlet.  That would seem to make it easier to write 
synchronous-looking code that operates on futures, and allow futures 
to be composed more cleanly.

ISTM that if futures were standardized before a coroutine API, it 
would lead to effective orphaning ala what happened with asyncore, 
especially since the new library is, well, new.

I'm somewhat concerned that, as described, the proposed API adds 
little over what's relatively easy to do with a mature coroutine 
framework like Eventlet, while at the same time creating yet another 
alternative (and mutually incompatible) event loop system in the 
stdlib, beyond the ones that are already in asyncore, tkinter, and 
the various SocketServer subclasses.

As far as naming goes, Twisted uses the term "Deferred" for this 
concept (and also has a very mature API for handling them).

And speaking of Twisted, it seems to me that the PEP would be much 
improved in general by learning from some of the lessons of other 
systems.  I don't think that Java's example is really the best one to 
follow in this instance, compared to the many existing async 
frameworks that have Python-specific experience and APIs to learn from.

Or, to put it another way, something that worries me about this PEP 
is that nearly all of its Python-related citations are for 
*discussions* of futures, with the only previous Python 
implementation cited being a crude sketch of a cookbook recipe.  The 
PEP also doesn't address questions of interoperability with existing 
solutions, compare features with them, or even so much as say, "There 
are other production implementations of this concept in Python, but 
we are going to pretend they don't exist."  ;-)