So attached (and at http://codereview.appspot.com/96125/show ) is a preliminary fix, correcting the problem with os.fork(), os.forkpty() and os.fork1(). This doesn't expose a general API for C code to use, for two reasons: it's not easy, and I need this fix more than I need the API change :-) (I actually need this fix myself for Python 2.4, but it applies fairly cleanly.)
To fix the mutex-across-fork problem correctly we should really acquire three locks (the import lock, the GIL and the TLS lock, in that order.) The import lock is re-entrant, and the TLS lock is tightly confined to the thread-local-storage lookup functions, but the GIL is neither re-entrant nor inspectable. That means we can't use pthread_atfork (we can't tell whether we have the GIL already or not, right before the fork), nor can we provide a convenient API for extension modules to use, really. The best we can do is provide three functions, pthread_atfork-style: a 'prepare' function, an 'after-fork-in-child' function, and an 'after-fork-in-parent' function. The 'prepare' function would start by releasing the GIL, then acquire the import lock, the GIL and the TLS lock in that order. It would also record *somewhere* the thread_ident of the current thread. The 'in-parent' function would simply release the TLS lock and the import lock, and the 'in-child' would release those locks, call the threading._at_fork() function, and fix up the TLS data, using the recorded thread ident to do lookups. The 'in-child' function would replace the current PyOS_AfterFork() function (which blindly reinitializes the GIL and the TLS lock, and calls threading._at_fork().)
Alternatively we could do what we do now, which is to ignore the fact that thread idents may change by fork() and thus that thread-local data may disappear, in which case the 'in-child' function could do a little less work. I'm suitably scared and disgusted of the TLS implementation, the threading implementations we support (including the pthread one) and the way we blindly type-pun a pthread_t to a long, that I'm not sure I want to try and build something correct and reliable on top of it.
Thomas Wouters <email@example.com>
Hi! I'm a .signature virus! copy me into your .signature file to help me spread!