[issue6721] Locks in python standard library should be sanitized on fork

[Charles-François Natali]

Charles-François Natali <neologix at free.fr> added the comment:

> Except for multiprocessing, does anyone know of any other module in the standard library that uses fork() and threads at the
> same time? After some grepping through the source I couldn't find any other cases.

The same problem arises in case of user-created threads, this problem
is not specific to the multiprocessing.

> Just to clarify: it's not that POSIX atfork() handlers run from signal handlers. It's that after a fork in a multithreaded process POSIX only guarantees calls to "safe" functions, which is the same set of functions as those that are safe to call from signal handlers. This fact does not change for Python's os.fork().
>

I think Nir knows perfectly that, he was just referring to a
limitation of pthread_atfork:
- fork() is async-safe, and thus can be called from a signal handler
- but if pthread_atfork handlers are installed, then fork() can become
non async-safe, if the handlers are not async-safe (and it's the case
when you're dealing with POSIX mutexes for example)
But since Python's user-defined signal handlers are actually called
synchronously (and don't run on behalf of the signal handler), there's
no risk of fork() being called from a signal handler.

> I'm still in favor of just deprecating using fork() on a multithreaded process (with appropriate warnings and documentation)

We can't do that, it would break existing code.
Furthermore, some libraries use threads behind the scene.

> I'm prepared to work on a patch that would remove the need for helper threads in the multiprocessing module.

What do you mean by helper threads?

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue6721>
_______________________________________