Why does this launch an infinite loop of new processes?

Robert Kern robert.kern at gmail.com
Thu Dec 22 06:24:21 EST 2011


On 12/21/11 8:11 PM, Andrew Berg wrote:
> On 12/21/2011 1:29 PM, Ethan Furman wrote:
>> Anything that runs at import time should be protected by the `if
>> __name__ == '__main__'` idiom as the children will import the __main__
>> module.
> So the child imports the parent and runs the spawn code again? That
> makes sense.

This is a problem with multiprocessing on Windows. Your code would work fine on 
a UNIX system. The problem is that Windows does not have a proper fork() for 
multiprocessing to use. Consequently, it has to start a new Python process from 
scratch and then *import* the main module such that it can properly locate the 
rest of the code to start. If you do not guard the spawning code with a __main__ 
test, then this import will cause an infinite loop.

Just as a further note on these lines, when older versions of setuptools and 
distribute install scripts, they generate stub scripts that do not use a 
__main__ guard, so installing a multiprocessing-using application with 
setuptools can cause this problem. I believe newer versions of distribute has 
this problem fixed, but I'm not sure.

-- 
Robert Kern

"I have come to believe that the whole world is an enigma, a harmless enigma
  that is made terrible by our own mad attempt to interpret it as though it had
  an underlying truth."
   -- Umberto Eco




More information about the Python-list mailing list