[Python-Dev] pthreads, fork, import, and execvp

[Guido van Rossum]

Yeah, I think imports inside functions are overused.

On 5/9/06, Rotem Yaari <vmalloc at gmail.com> wrote:
> Hello everyone!
>
> We have been encountering several deadlocks in a threaded Python
> application which calls  subprocess.Popen (i.e. fork()) in some of its
> threads.
>
> This has occurred on Python 2.4.1 on a 2.4.27 Linux kernel.
>
>  Preliminary analysis of the hang shows that the child process blocks
> upon entering the  execvp function, in which the import_lock is acquired
> due to the following line:
>
> def _ execvpe(file,  args,  env=None):
>     from  errno import ENOENT, ENOTDIR
>     ...
>
> It is known that when forking from a  pthreaded application, acquisition
> attempts on locks which were already locked by other threads while
> fork() was called will deadlock.
>
> Due to these oddities we were wondering if it would be better to extract
> the above import line from the  execvpe call, to prevent lock
> acquisition attempts in such cases.
>
> Another workaround could be re-assigning a new lock to import_lock
> (such a thing is done with the global interpreter lock) at PyOS_AfterFork or
> pthread_atfork.
>
> We'd appreciate any opinions you might have on the subject.
>
> Thanks in advance,
>
> Yair and  Rotem
> _______________________________________________
> Python-Dev mailing list
> Python-Dev at python.org
> http://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe: http://mail.python.org/mailman/options/python-dev/guido%40python.org
>


-- 
--Guido van Rossum (home page: http://www.python.org/~guido/)