[issue27126] Apple-supplied libsqlite3 on OS X is not fork safe; can cause crashes

[Ned Deily]

Ned Deily added the comment:

Thanks for your analysis!  I'm sure that you are correct about Issue20353.  It could also be the root cause of other crashes reported when internet proxies are used with urllib* invoking the _scproxy helper extension module, as reported in Issue13829.

The question is what can be done.  There seem to be two separate issues here.  The first is a crash when using Python's sqlite3 module, the problem your test case files demonstrate.  Note this *isn't* a problem when using the sqlite3 module from any of the current python.org OS X binary installer Pythons (2.7 or 3.x) because those Pythons do not use the Apple-supplied system libsqlite3; they already link with their own newer private copy.  Your py2 test, of course, does fail when using the Apple-supplied system Pythons.  We can't solve that problem; only Apple can; you could open a bug report with them.  If other distributors of Python on OS X rely on the system libsqlite3, they could avoid such crashes by also supplying their own copy.  For example, MacPorts already does, so your test cases don't fail with their Pythons.  I don't know what other distributors, like Homebrew or Anaconda, do.  We could add a note to Mac/README.

Second, the stickier (and totally separate) problem is what to do about _scproxy.  If, under the covers, the calls that _scproxy make to the Apple-supplied System Configuration framework use the un-forksafe Apple libsqlite3, there is nothing we can do about that; supplying a private copy of libsqlite3 isn't going to change what the framework uses and, in any case, it would be a really bad idea to even try to hack that.  So, if we don't change _scproxy or urllib*'s use of it, only Apple can fix the problem.  Since we can't expect that that is going to happen, the question becomes what alternatives are there.  One would be to find a way to eliminate _scproxy or its use of the unsafe SC framework calls.  Another approach would be to simply document ths restriction that urllib calls invoking ProxyHandler must be made in a main process (or whatever the precise restriction is) and leave it at that (https://docs.python.org/dev/library/urllib.request.html#urllib.request.ProxyHandler).

Ronald, what do you think?

----------
title: Mac system sqlite3 not fork safe: Bundle a version? -> Apple-supplied libsqlite3 on OS X is not fork safe;  can cause crashes
type:  -> crash
versions: +Python 3.6

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue27126>
_______________________________________