[Python-Dev] Issues with multiprocessing

[Michael Foord]

On 09/12/2009 08:51, Joshi, Yateen wrote:
>
> Hi,
>
> I have an application that uses multiprocessing pools 
> (multiprocessing.Pool(processes=.....)). There are multiple such pools 
> and each pool has configurable number of processes. Once the process 
> is spawned, it keeps on executing and does the needed processing. If 
> there is nothing to process (like ftp'ing files from some source, if 
> the files are not there, the process would sleep for some time, and 
> then again check for files, that way, it is a infinite loop with some 
> sleep), the process 'sleeps' for some time and continues.
>
> I am using a T5220, Solaris box with Solaris 10.
>
> Problem --there are multiple pools and multiple processes, i am seeing 
> that not all the processes get spawned. They get spawned when the 
> sleep time is increased (say from 0.1 sec to 1 sec). If I further 
> increase the number of processes, again some process do not get 
> spawned. For that, I further need to increase the sleep time (say to 2 
> sec), then the processes get spawned.
>
> Typically, in a multiprocessing I would expect that if a process 
> sleeps for even a small time, other processes should get their chance 
> to execute, but this does not seem to be happening here. Can you 
> please throw some light on it?
>

Hello Joshi,

This email list is for the development of Python, not for developing 
with Python. A better place to try would be the python-list 
(comp.lang.python - which has both google groups and gmane gateways).

All the best,

Michael Foord


> Thanks and Regards,
>
> Yateen V. Joshi
>
> This email and any attachments may contain legally privileged and/or 
> confidential information of Starent Networks, Corp. and is intended 
> only for the individual or entity named in the message. The 
> information transmitted may not be used to create or change any 
> contractual obligations of Starent Networks, Corp. Any review, 
> retransmission, dissemination or other use of, or taking of any action 
> in reliance upon this e-mail and its attachments by persons or 
> entities other than the intended recipient is prohibited. If you are 
> not the intended recipient, please notify the sender immediately -- by 
> replying to this message or by sending an email to 
> postmaster at starentnetworks.com -- and destroy all copies of this 
> message and any attachments without reading or disclosing their 
> contents. Thank you.
>
>
> _______________________________________________
> Python-Dev mailing list
> Python-Dev at python.org
> http://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe: http://mail.python.org/mailman/options/python-dev/fuzzyman%40voidspace.org.uk
>    


-- 
http://www.ironpythoninaction.com/
http://www.voidspace.org.uk/blog


-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20091209/b859660f/attachment.htm>