[Python-Dev] PEP 3148 ready for pronouncement

[Scott Dial]

On 5/27/2010 4:13 AM, Brian Quinlan wrote:
> On 27 May 2010, at 17:53, Floris Bruynooghe wrote:
>> On Thu, May 27, 2010 at 01:46:07PM +1200, Greg Ewing wrote:
>>> On 27/05/10 00:31, Brian Quinlan wrote:
>>>> You have two semantic choices here:
>>>> 1. let the interpreter exit with the future still running
>>>> 2. wait until the future finishes and then exit
>>>
>>> I'd go for (1). I don't think it's unreasonable to
>>> expect a program that wants all its tasks to finish
>>> to explicitly wait for that to happen.
>>
>> I'd got for (1) as well, it's no more then reasonable that if you want
>> a result you wait for it.  And I dislike libraries doing magic you
>> can't see, I'd prefer if I explicitly had to shut a pool down.
>>
>> I'm glad I'm not alone in preferring (1) tough.
> 
> Keep in mind that this library magic is consistent with the library
> magic that the threading module does - unless the user sets
> Thread.daemon to True, the interpreter does *not* exit until the thread
> does.

Given your rationale, I don't understand from the PEP:

> shutdown(wait=True)
> 
> Signal the executor that it should free any resources that it is
> using when the currently pending futures are done executing. Calls to
> Executor.submit and Executor.map and made after shutdown will raise
> RuntimeError.
> 
> If wait is True then the executor will not return until all the
> pending futures are done executing and the resources associated with
> the executor have been freed.

Can you tell me what is the expected execution time of the following:

>>> executor = ThreadPoolExecutor(max_workers=1)
>>> executor.submit(lambda: time.sleep(1000))
>>> executor.shutdown(wait=False)
>>> sys.exit(0)

I believe it's 1000 seconds, which seems to defy my request of
shutdown(wait=False) because "secretly" the Python exit is going to wait
anyways. ISTM, it is much easier to get behavior #2 if you have behavior
#1, and it would also seem rather trivial to make ThreadPoolExecutor
take an optional argument specifying which behavior you want.

Your reference implementation does not actually implement the
specification given in the PEP, so it's quite impossible to check this
myself. There is no wait=True option for shutdown() in the reference
implementation, so I can only guess what that implementation might look
like.

-- 
Scott Dial
scott at scottdial.com
scodial at cs.indiana.edu