Pool module -- does map pre-assign tasks to processes?
aahz at pythoncraft.com
Mon Oct 5 20:14:35 CEST 2009
In article <a6593d46-0f0e-4ca7-8481-c53b18218f57 at o13g2000vbl.googlegroups.com>,
Luca <luca.de.alfaro at gmail.com> wrote:
>I would like to use the Pool module, but my tasks take sometimes
>unpredictably different time to complete. The simplest way to write
>the code would be to put all task descriptions in an array, then call
>p = Pool(8)
>But I don't want to preassign process 0 with elements a, a, a
>, process 1 with elements a, a, a, and so forth.
>Rather, I would like all tasks to be put into a queue, and I would
>like the processes to each time grab the next task to be done, and do
>it. This would ensure fairly equal loading.
>My question is: does the map() method of Pool pre-assign which task
>gets done by each process, or is this done at runtime, on a get-first-
You can probably get what you want by setting chunksize to 1.
Aahz (aahz at pythoncraft.com) <*> http://www.pythoncraft.com/
"....Normal is what cuts off your sixth finger and your tail..." --Siobhan
More information about the Python-list