[Python-ideas] An error in multiprocessing on MacOSX?
Andrew Barnert
abarnert at yahoo.com
Wed Nov 21 21:25:01 CET 2012
From: Antoine Pitrou <solipsis at pitrou.net>
To: python-ideas at python.org
>
> On Wed, 21 Nov 2012 16:12:04 +0100
> Sturla Molden <sturla at molden.no> wrote:
>
> > According to Apple enineers:
> >
> > """
> > For API outside of POSIX, including GCD and technologies like
> > Accelerate, we do not support usage on both sides of a fork(). For
> > this reason among others, use of fork() without exec is discouraged in
> > general in processes that use layers above POSIX.
> > """
> >
> > Multiprocessing on OSX calls os.fork, but not os.exec.
> >
> > Thus, is multiprocessing errorneously implemented on Mac? Forking
> > without calling exec means that only APIs inside POSIX can be used by
> > the child process.
>
> Or perhaps "fork()" is erroneously implemented on Mac.
No, it's not that fork is erroneously implemented, it's that CoreFoundation, as
designed, doesn't work across a properly-implemented POSIX fork. And presumably
the same is true for various other Apple technologies, but they won't give a
complete list of what is and isn't safe and why—instead, they just say "don't
use any non-POSIX stuff if you fork without exec".
And the problem Python users face isn't that multiprocessing works differently
on OS X vs. FreeBSD or linux, but that their programs may be quietly using
non-fork-safe things like Accelerate.framework on OS X but not on FreeBSD or
linux.
More information about the Python-ideas
mailing list