[stdlib-sig] futures - a new package for asynchronous execution

Brian Quinlan brian at sweetapp.com
Sat Nov 28 03:18:37 CET 2009

Hey Antoine,

Sorry for not getting back to you sooner - I actually thought that I  
did reply but I see now that I didn't.

On 14 Nov 2009, at 01:22, Antoine Pitrou wrote:

> Hey,
>> Future remains unchanged - I disagree that Deferreds would be better,
>> that .exception() is not useful, and that .result() should be
>> renamed .get() or .__call__().
> On what grounds do you disagree with the latter?

It feels hacky. Getting the result doesn't feel so special that it  
deserves to be a call rather than a simple getter.

> Another question: is the caught exception an attribute of the future?


> If
> so, is there any mechanism to clean it up (and its traceback) once the
> future has been "consumed"?

No there isn't. That's a good point though. I wonder if futures will  
tend to long-lived after there results are available?

>> map becomes a utility function:
>> def map(executor, *iterables, timeout=None)
> Why? map() can be defined on the ABC, so that subclasses don't have to
> provide their own implementation.
> An utility function which looks like a method and shadows the name  
> of a
> built-in looks like a bad choice to me.

Good point.

>> wait becomes a utility function that can wait on any iterable of
>> Futures:
>> def wait(futures, return_when=ALL_COMPLETED)
> Does it work if the futures are executed by different executors?
> If not, it should be an Executor method.

Yes, it goes.

>> return_when indicates when the method should return. It must be one  
>> of
>> the following constants:
> Can you outline the difference between NEXT_COMPLETED and
> NEXT_EXCEPTION? What happens if I ask for NEXT_COMPLETED but the next
> future to complete raises an exception? etc.

NEXT_COMPLETED includes futures that raise. Completed in this sense  
means "done running".

>> def itercompleted(futures, timeout=None):
>> Returns an iterator that returns a completed Future from the given
>> list when __next__() is called. If no Futures are completed then
>> __next__() is called then __next__() waits until one does complete.
> What about futures which complete with an exception?

They are included.

>> with futures.ThreadPoolExecutor(50) as executor:
>>   fs = [executor.submit(load_url, url, timeout=30) for url in URLS]
> The use of "with" here still is counter-intuitive, because it does not
> clean up resources immediately as it would seem to do. "with" is  
> always
> synchronous in other situations.

Maybe waiting until all pending futures are done executing would be  

>> What do you think? Are we moving in the right direction?
> Perhaps, yes, but there are still lots of dark areas.
> Besides, it's obvious that the package has to mature, and should be
> tested by other people.

It would be great if other people tested the API. I'm not sure what  
you mean by "mature" though.


More information about the stdlib-sig mailing list