Get multiprocessing.Queue to do priorities

uuid M8R-gqmgb4 at
Sat May 9 18:11:21 EDT 2009

The Queue module, apparently, is thread safe, but *not* process safe. 
If you try to use an ordinary Queue, it appears inaccessible to the 
worker process. (Which, after all, is quite logical, since methods for 
moving items between the threads of the same process are quite 
different from inter-process communication.) It appears that creating a 
manager that holds a shared queue might be an option 


for illustration: This shows that Queue.Queue doesn't work with processes:

def worker(queue):
    while True:
        item = queue.get()
        print item

queue_queue = Queue.Queue()

worker_thread = multiprocessing.Process(target=worker, args=(queue_queue,))
for i in range(10):
while True:
        print 'still on queue: ' + queue_queue.get(False)
    except Queue.Empty:

This yields:

still on queue: 0
still on queue: 1
still on queue: 2
still on queue: 3
still on queue: 4
still on queue: 5
still on queue: 6
still on queue: 7
still on queue: 8
still on queue: 9

So no queue item ever arrives at the worker process.

On 2009-05-09 22:00:36 +0200, Scott David Daniels <Scott.Daniels at Acm.Org> said:

> 2.6 has a PriorityQueue in the Queue module.
> If you aren't using 2.6, you could copy the code for your own version.

More information about the Python-list mailing list