[Python-Dev] Issues with multiprocessing

[Jesse Noller]

On Wed, Dec 9, 2009 at 6:22 AM, Michael Foord <fuzzyman at voidspace.org.uk> wrote:
> On 09/12/2009 08:51, Joshi, Yateen wrote:
>
> Hi,
>
>
>
> I have an application that uses multiprocessing pools
> (multiprocessing.Pool(processes=…..)). There are multiple such pools and
> each pool has configurable number of processes. Once the process is spawned,
> it keeps on executing and does the needed processing. If there is nothing to
> process (like ftp’ing files from some source, if the files are not there,
> the process would sleep for some time, and then again check for files, that
> way, it is a infinite loop with some sleep), the process ‘sleeps’ for some
> time and continues.
>
>
>
> I am using a T5220, Solaris box with Solaris 10.
>
> Problem –there are multiple pools and multiple processes, i am seeing that
> not all the processes get spawned. They get spawned when the sleep time is
> increased (say from 0.1 sec to 1 sec). If I further increase the number of
> processes, again some process do not get spawned. For that, I further need
> to increase the sleep time (say to 2 sec), then the processes get spawned.
>
>
>
> Typically, in a multiprocessing I would expect that if a process sleeps for
> even a small time, other processes should get their chance to execute, but
> this does not seem to be happening here. Can you please throw some light on
> it?
>
> Hello Joshi,
>
> This email list is for the development of Python, not for developing with
> Python. A better place to try would be the python-list (comp.lang.python -
> which has both google groups and gmane gateways).
>
> All the best,
>
> Michael Foord
>


Additionally, if you suspect there is a reproducible box (preferably
not just on solaris, which I do not have access to) please file it in
the tracker.