[Distutils] Does anyone understand what's going on with libpython on Linux?

[Matthew Brett]

On Sun, Feb 7, 2016 at 2:19 PM, Robert T. McGibbon <rmcgibbo at gmail.com> wrote:
>> One option we have then is to remove all DT_NEEDED references to
> libpython in manylinux wheels. We get instant compatibility for bare
> Debian / Ubuntu Python installs, at the cost of causing some puzzling
> crash for the case of: dlopened library with embedded Python
> interpreter where the embedded Python interpreter imports a manylinux
> wheel.
>
> I don't think this is acceptable, since it's going to break some packages
> that depend
> on dlopen.
>
>> On the other hand, presumably this same crash will occur for nearly
> all Debian-packaged Python extension modules (if it is true that they
> do not specify a libpython dependency) - so it seems unlikely that
> this is a common problem.
>
> I don't think so. Debian-packaged extensions that require libpython to exist
> (a minority of them to be sure, but ones that use complex shared library
> layouts)
> just declare a dependency on libpython. For example, python-pyside has a
> Depends on libpython2.7:
>
> ```
> $ apt-cache depends python-pyside.qtcore
> python-pyside.qtcore
>   Depends: libc6
>   Depends: libgcc1
>   Depends: libpyside1.2
>   Depends: libpython2.7
>   Depends: libqtcore4
>   Depends: libshiboken1.2v5
>   Depends: libstdc++6
>   Depends: python
>   Depends: python
>   Conflicts: python-pyside.qtcore:i386
> ```

Sure - and this might be because the pyside packager was being
especially careful about libpython, or it might be an accident -
pyside is hard to build.

On the other hand, it looks like almost all the common Debian packages
don't declare this dependency - so almost all of the standard
scientific Python stack and more would crash in this corner case:

apt-cache depends python-numpy | grep libpython
apt-cache depends python-scipy | grep libpython
apt-cache depends python-yaml | grep libpython
apt-cache depends python-regex | grep libpython
apt-cache depends python-matplotlib | grep libpython

It seems reasonable to build to the same compatibility level as most
Debian packaged modules.

Matthew