multiprocessing.Pool, its queue, and pre-emption
ladasky at my-deja.com
Fri Sep 16 08:31:49 CEST 2011
On Sep 15, 1:52 pm, John Ladasky <lada... at my-deja.com> wrote:
> I've been snooping around inside Pool, and I would guess that what I
> want to do is to manipulate Pool._inqueue, which is a
> multiprocessing.queues.SimpleQueue object. I haven't found any
> documentation for SimpleQueue. It appears to have only the most
> rudimentary of public methods -- put, get, and empty.
Reading more deeply, I think that my first guess was incorrect.
There's also Pool._taskqueue, and it's a full-fledged Queue. It
appears that map.async() puts pending jobs there, not in
If this is true, then all that should have to be done is to override a
few methods. I'm going to try it.
More information about the Python-list