Issues with multiprocessing

[Joshi, Yateen]

Hi,

 

I have an application that uses multiprocessing pools
(multiprocessing.Pool(processes=.....)). There are multiple such pools
and each pool has configurable number of processes. Once the process is
spawned, it keeps on executing and does the needed processing. If there
is nothing to process (like ftp'ing files from some source, if the files
are not there, the process would sleep for some time, and then again
check for files, that way, it is a infinite loop with some sleep), the
process 'sleeps' for some time and continues.

 

I am using a T5220, Solaris box with Solaris 10. 

Problem -there are multiple pools and multiple processes, i am seeing
that not all the processes get spawned. They get spawned when the sleep
time is increased (say from 0.1 sec to 1 sec). If I further increase the
number of processes, again some process do not get spawned. For that, I
further need to increase the sleep time (say to 2 sec), then the
processes get spawned.

 

Typically, in a multiprocessing I would expect that if a process sleeps
for even a small time, other processes should get their chance to
execute, but this does not seem to be happening here. Can you please
throw some light on it?

 

Thanks and Regards,

 

Yateen V. Joshi

 

 


This email and any attachments may contain legally privileged and/or confidential information of Starent Networks, Corp. and is intended only for the individual or entity named in the message.  The information transmitted may not be used to create or change any contractual obligations of Starent Networks, Corp.  Any review, retransmission, dissemination or other use of, or taking of any action in reliance upon this e-mail and its attachments by persons or entities other than the intended recipient is prohibited. If you are not the intended recipient, please notify the sender immediately -- by replying to this message or by sending an email to postmaster at starentnetworks.com -- and destroy all copies of this message and any attachments without reading or disclosing their contents. Thank you.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-list/attachments/20091209/d07da39e/attachment.html>