[Python-Dev] pthreads, fork, import, and execvp

[Nick Coghlan]

Rotem Yaari wrote:
> This has occurred on Python 2.4.1 on a 2.4.27 Linux kernel.
> 
>  Preliminary analysis of the hang shows that the child process blocks
> upon entering the  execvp function, in which the import_lock is acquired
> due to the following line:
> 
> def _ execvpe(file,  args,  env=None):
>     from  errno import ENOENT, ENOTDIR
>     ...

While I agree there's no good reason to embed this import inside the function, 
I also don't see how you could be getting a deadlock unless you have modules 
that either spawn new threads or fork the process as a side effect of being 
imported.

Neither of those is guaranteed to work properly due to the fact that the 
original thread holds the import lock, leading to the possibility of deadlock 
if the spawned thread tries to import anything (as seems to be happening here) 
and the original thread then waits for a result from the spawned thread.

Spawning new threads or processes can really only be safely done when invoked 
directly or indirectly from the __main__ module (as that is run without 
holding the import lock). When done as a side effect of module import, the 
module needs to ensure that the result of the spawned thread or process is 
only waited for after the original module import is complete (and the import 
lock released as a result).

Cheers,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia
---------------------------------------------------------------
             http://www.boredomandlaziness.org