[Python-Dev] [PEP 3148] futures - execute computations asynchronously

[Jeffrey Yasskin]

On Sun, Mar 7, 2010 at 9:57 AM, P.J. Eby <pje at telecommunity.com> wrote:
> At 08:39 AM 3/7/2010 -0800, Jeffrey Yasskin wrote:
>> Do you have an example of a language or library that uses the term
>> "future" to refer to what you're talking about? I'm curious to see
>> what it looks like.
>
> The wikipedia page menetioned earlier in the discussion has several
> examples.  Twisted's Deferreds are probably the best example of such a
> system in Python, though, as far as I know.  The features proposed in the
> PEP are basically akin to Twisted's deferToThread(), i.e. "return a future
> for running this code in some other thread".

Nearly all of the wikipedia page's examples refer to exactly this kind
of system, but with syntax to support it instead of putting it in a
library. The distinction between futures and deferreds is that futures
are designed to block, while deferreds are designed to continue
through callbacks. We could say that futures are to deferreds as
threading.Lock is to mutex.mutex.

> That way you
> can write synchronous-looking code that nonetheless executes asynchronously.

 "Asynchronous" in the PEP refers to how the task behind the future
runs, not how users wait for the future. Is there a better way to say
that?

> Twisted uses Deferreds extensively, both for I/O and IPC, and of course for
> database access.  However, the 'defer' module itself is conceptually
> distinct from both the I/O system, event loop, and threads: it's a
> largely-generic framework for managing asynchronous computation.  Parallel
> task queueing is simply *one* possible application of Deferreds; their
> actual purpose is to provide a common API for working with values that
> aren't yet known.
>
> That is to say, futures.
>
> In contrast, the PEP manages only a very limited sort of "future" - the
> completion of a single task, which must be synchronously waited for.  If one
> "future" in the PEP needs to wait for another, you tie up that thread or
> process in a busy loop...  which rather limits how much parallelism you can
> have.

So is it that you just don't like the idea of blocking, and want to
stop anything that relies on it from getting into the standard
library?

Given the set_result and set_exception methods, it's pretty
straightforward to fill in the value of a future from something that
isn't purely computational. Given a way to register "on-done"
callbacks with the future, it would be straightforward to wait for a
future without blocking, too.

Jeffrey