[issue9634] Add timeout parameter to Queue.join()

[Nick Coghlan]

Nick Coghlan <ncoghlan at gmail.com> added the comment:

I just created #14487 to request a documented API (Queue.pending()) that provides a formal mechanism for asking whether or not any jobs are still pending (analagous to the existing empty() and full() query methods).

Specifically, what I have is a client process that is executed periodically, gathers up a set of tasks and uses a thread pool to submit them in parallel to a synchronous API.

If all tasks complete with no new tasks being scheduled, then the client should terminate. However, if a new task arrives while any existing task is still in progress, then the client should submit it "immediately" (where, due to the time scales involved, "immediately" actually means "within the next few minutes" so I have plenty of scope to let the client block for a while instead of implementing a busy loop).

So, a timeout on join() would actually fit my use case better than the pending() API I proposed in the other issue. The processing loop would then look something like:

while have_tasks_to_process():
   submit_tasks_to_queue()
   try:
       task_queue.join(timeout)
   except Pending:
       pass

The advantage of having the timeout is that it would avoid the clumsy workarounds needed to avoid the busy loop created by the use of a query based approach. (Of course, I'm going to have to use the workaround anyway, since my client runs on Python 2.6, but still, I believe it meets the "concrete use case" criterion).

>From a design aesthetic point of view, a pending() query API and a timeout on join() that throws a Pending exception would match the existing empty()/get()/Empty and full()/put()/Full API triples.

----------
nosy: +ncoghlan
resolution: rejected -> 
status: closed -> open

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue9634>
_______________________________________