[Python-ideas] An error in multiprocessing on MacOSX?

[Gregory P. Smith]

On Wed, Nov 21, 2012 at 11:34 AM, Mike Meyer <mwm at mired.org> wrote:

> On Wed, Nov 21, 2012 at 9:12 AM, Sturla Molden <sturla at molden.no> wrote:
> > See this:
> >
> > http://mail.scipy.org/pipermail/numpy-discussion/2012-August/063593.html
> >
> > According to Apple engineers:
> >
> > """
> > For API outside of POSIX, including GCD and technologies like
> > Accelerate, we do not support usage on both sides of a fork(). For
> > this reason among others, use of fork() without exec is discouraged in
> > general in processes that use layers above POSIX.
> > """
> >
> > Multiprocessing on OSX calls os.fork, but not os.exec.
> >
> > Thus, is multiprocessing errorneously implemented on Mac? Forking without
> > calling exec means that only APIs inside POSIX can be used by the child
> > process.
>

I don't care how this is read or what the default behavior is. All I want
is for someone who cares about multiprocessing actually working well for
people to implement optional support for *not* using os.fork() on posixish
systems.  ie: port an equivalent of the windows stuff over.

I don't use multiprocessing so I've never looked into adding it.  It
shouldn't be difficult given the legwork has already been done for use on
windows.  this is exactly what issue8713 is asking for.


> > Does multiprocessing needs a reimplementation on Mac to behave as it
> does on
> > Windows? (Yes it would cripple it similarly to the crippled
> multiprocessing
> > on Windows.)
>
> -1.
>

+10 - though I wouldn't call it a "re" implementation, just a port of the
windows stuff.

Mac's make nice Unix desktops (if you like their GUI), and it's not
> uncommon to find people writing & testing software on Mac's for
> deployment to POSIX servers. Such people using the multiprocessing
> module need it to provide proper POSIX behavior.
>

Those people are delusional, they need to test their software on the OS it
is destined to run on. Not on their terminal.


> In general, these issues are simply a case of needing to be aware of
> what your program is doing, and making sure you don't do things that
> can cause problems - whether you're using above-the-POSIX Mac APIs, or
> POSIX the problematic Posix APIs.  A case might be made that the
> multiprocessing module should help with that, by providing a way to
> say "I'm doing things that make fork dangerous, please use the
> system-appropriate fork+exec call instead."
>

Yep.  Like I said, I don't personally care what the default is.  I just
want support for both options so that people have a way to _not_ shoot
themselves in the foot.

-gps
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20121121/a4429eeb/attachment.html>