[issue34884] Python may load incorrect libraries when embedding the macOS system python 2.7

[Ned Deily]

Ned Deily <nad at python.org> added the comment:

Ugh!  That's a messy one.  Just for the record, the problem of searching other than the system Python images is reproducible with other distributions of Python, including the python.org Python 2.7 - so it is not limited to Homebrew.  The key seems to be which "python" (or possibly "python2.7") is found first on $PATH.  It also appears to not be limited to framework builds of Python; a non-framework build of 2.7 inserted first on $PATH also caused a failure.  I took a quick look at how lldb apparently invokes python, following the link in the StackOverflow item referenced in the llvm bug tracker issue:

http://llvm.org/svn/llvm-project/lldb/trunk/source/Plugins/ScriptInterpreter/Python/ScriptInterpreterPython.cpp

and I'm not going to try to try to step through all that.

Much of the magic that's involved in initializing Python search paths is in Modules/getpath.c.  In it, you'll see that we do some magic using long deprecated macOS dyld APIs, namely NSModuleForSymbol, NSLookupAndBindSymbol, and NSLibraryNameForModule.  It's quite possible that the behavior seen is due to how they search for symbols.  One could try to track them down in the dyld sources found in https://opensource.apple.com.

But now that I think of it, we have had an open issue (Issue15498) for a long time to avoid the use of those deprecated APIs and, lo and behold, one of the comments under it (msg173828) might very well be the key to what's going on here:

"Apple seems to have switched to using dlopen in their version of getpath.c (see <http://www.opensource.apple.com/source/python/python-60.3/2.7/fix/getpath.c.ed>, this is the version in OSX 10.8.2)

This causes a problem for some people, as noticed on the pythonmac mailing list. This is something we should test before applying my patch to avoid regressions. [...]"

If so, the problem may have to be fixed by Apple in the system Python.

Ronald undoubtedly has better insight into this.

----------
components:  -Library (Lib)
title: Python loads incorrect libraries on MacOS, 2.7 -> Python may load incorrect libraries when embedding the macOS system python 2.7

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue34884>
_______________________________________