On Wed, Jun 24, 2015 at 12:31 PM, Gregory P. Smith <greg@krypto.org> wrote:
You cannot assume that fork() is safe on any OS as a general solution for anything. This isn't a Windows specific problem, It simply cannot be relied upon in a general purpose library at all. It is incompatible with threads.
The ways fork() can be used safely are in top level application decisions: There must be a guarantee of no threads running before all forking is done. (thus the impossibility of relying on it as a mechanism to do anything useful in a generic library - you are a library, you don't know what the whole application is doing or when you were called as part of it)
A concurrency model that assumes that it is fine to fork() and let child processes continue to execute is not usable by everyone. (ie: multiprocessing until http://bugs.python.org/issue8713 was implemented).
Another way of looking at it is that a concurrency model that assumes it is fine to thread and let child threads continue to execute is not usable by everyone. IMO the lesson here is don't start threads *or* fork processes behind the scenes without explicitly allowing your callers to override you, so that the top level app can orchestrate everything appropriately. This is especially important in Python, where forking is one of the best ways of getting single-machine multicore processing. Interestingly, the worker threads in OP can probably be made fork-safe. Not sure that's especially useful, but I can imagine. -- Devin