[Python-ideas] An error in multiprocessing on MacOSX?

[Andrew Barnert]

From: Antoine Pitrou <solipsis at pitrou.net>

To: python-ideas at python.org
> 
> On Wed, 21 Nov 2012 16:12:04 +0100
> Sturla Molden <sturla at molden.no> wrote:
> 
> > According to Apple enineers:
> > 
> > """
> > For API  outside of POSIX, including GCD and technologies like
> > Accelerate, we do  not support usage on both sides of a fork(). For
> > this reason among  others, use of fork() without exec is discouraged in
> > general in  processes that use layers above POSIX.
> > """
> > 
> >  Multiprocessing on OSX calls os.fork, but not os.exec.
> > 
> > Thus, is  multiprocessing errorneously implemented on Mac? Forking 
> > without  calling exec means that only APIs inside POSIX can be used by 
> > the child  process.
> 
> Or perhaps "fork()" is erroneously implemented on  Mac.

No, it's not that fork is erroneously implemented, it's that CoreFoundation, as 
designed, doesn't work across a properly-implemented POSIX fork. And presumably 
the same is true for various other Apple technologies, but they won't give a  
complete list of what is and isn't safe and why—instead, they just say "don't 
use any non-POSIX stuff if you fork without exec".

And the problem Python users face isn't that multiprocessing works differently 
on OS X vs. FreeBSD or linux, but that their programs may be quietly using 
non-fork-safe things like Accelerate.framework on OS X but not on FreeBSD or 
linux.