[Python-Dev] futures API

[Nick Coghlan]

On Sat, Dec 11, 2010 at 6:07 AM, Brian Quinlan <brian at sweetapp.com> wrote:
>> The problem also occurs when using a callback:
>> http://www.freehackers.org/~tnagy/futures_test2.py
>>
>> If it is necessary to catch KeyboardInterrupt exceptions to cancel the
>> futures execution, then how about adding this detail to the docs?
>
> AFAIK, catching KeyboardInterrupt exceptions is not sufficient.

finally blocks and with statements can get you a fairly long way,
though. futures does everything right on this front, as far as I can
see.

In this case, the problem is in the design of the test program. It
*must* get exactly as many items in the queue as were submitted to the
executor. If one is ever missing (e.g. due to a poorly timed
KeyboardInterrupt in the callback, as clearly happened in the
presented trace), it will hang in the final call to self.out_q.get().
There's a reason Queue.get offers both nonblocking and
block-with-timeout functionality. (Alternately, the management of
out_q and count could be made smarter, such that an exception in
adding a result to out_q triggered an appropriate adjustment in the
count of expected values)

I also agree with Brian that the variable naming for the sample code
and comments like "may use futures of different max_workers count. I
think it is not too far-fetched to create a new futures object each
time" suggests a fundamental confusion of the terms "future" and
"executor". Executors are not futures - the return value from an
executor's submit method is a future.

Cheers,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia