[issue27126] Apple-supplied libsqlite3 on OS X is not fork safe; can cause crashes

[Ronald Oussoren]

Ronald Oussoren added the comment:

There is a clear gap between two use cases for Python on OSX:

1) Integrate nicely with Apple stuff

2) Be useful as a development platform for software that will be deployed on Linux machines.

Both use cases are valid, but have slightly different trade-offs. I have both use-cases, but tend not to do things that cause problems here.  In any case I tend to favour the first use case, it is easy enough to develop locally and test on Linux (VMs, docker, ...) for the second use case.

A particular problem is using os.fork, as Evan notes there are problems with using Apples frameworks in programs that use fork without exec-ing a new program. Libdispatch is one problem, but higher-level frameworks are also problematic (the _scproxy problem is not only caused the use of libdispatch).

Spawning off a small executable for the _scproxy case could be worthwhile, although I haven't fully thought about the implications of that. I wouldn't like removing the use of _scproxy, it uses the documented programmatic way to access system-wide proxy configuration. Using something like scutil(1) might work as well, but is hackish.

Changing the default mode for multiprocessing could also be useful, but could result in calls about code that works on Linux but doesn't on OSX due to differences in fork mode.

BTW. Using fork without exec is unsafe in any program using threads, people tend to run into this more on OSX because Apple's libraries use background threads while other Unix-y platforms don't tend to do that.

As to shipping the sqlite with the CPython source: that's not something we do in general. It would be better to document the issue in the build instructions for CPython, and/or tweak the CPython build process to issue a warning when it detects that is using the system version of sqlite on OSX.

BTW2. Another problematic issue on OSX is accessing the system trust store for SSL certificates.  AFAIK the current binaries still use Apple's build of OpenSSL that uses a private API to access the system trust store, using a standalone OpenSSL would require some way to make OpenSSL use the trust store; either by shipping a script to dump the trust store in a format that OpenSSL can use or by using Apple's APIs to access the trust store. IIRC there is an issue about that, but I cannot find it at the moment. Using the public APIs might result in similar problems as mentioned in this issue.

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue27126>
_______________________________________