multiprocessing.Pool, its queue, and pre-emption

John Ladasky ladasky at my-deja.com
Fri Sep 16 08:31:49 CEST 2011


On Sep 15, 1:52 pm, John Ladasky <lada... at my-deja.com> wrote:
> I've been snooping around inside Pool, and I would guess that what I
> want to do is to manipulate Pool._inqueue, which is a
> multiprocessing.queues.SimpleQueue object.  I haven't found any
> documentation for SimpleQueue.  It appears to have only the most
> rudimentary of public methods -- put, get, and empty.

Reading more deeply, I think that my first guess was incorrect.
There's also Pool._taskqueue, and it's a full-fledged Queue.  It
appears that map.async() puts pending jobs there, not in
Pool._inqueue.

If this is true, then all that should have to be done is to override a
few methods.  I'm going to try it.



More information about the Python-list mailing list