[Distutils] draft PEP: manylinux2

[Mark Williams]

On Mon, Feb 05, 2018 at 03:17:55PM -0600, Jonathan Helmus wrote:
> On 02/03/2018 02:11 AM, Nathaniel Smith wrote:
> > > Docker Images
> > > -------------
> > > 
> > > ``manylinux2`` Docker images based on CentOS 6.9 x86_64 and i686 are
> > > provided for building binary ``linux`` wheels that can reliably be
> > > converted to ``manylinux2`` wheels.  [8]_ These images come with a
> > > full compiler suite installed (``gcc``, ``g++``, and ``gfortran``
> > > 4.8.2) as well as the latest releases of Python and  ``pip``.
> > We can and should use newer compiler versions than that, and probably
> > upgrade them again over the course of the image's lifespan, so let's
> > just drop the version numbers from the PEP entirely. (Maybe s/6.9/6/
> > as well for the same reason.)
> > 
> 
> Moving to GCC 5 and above will introduced the new libstd++ ABI. [1]  The
> manylinux2 standard need to define which ABI compiled libraries should be
> compiled against as older version of libstdc++ will not support the new
> ABI.  From what I recall the devtoolset packages for CentOS can only target
> the older, _GLIBCXX_USE_CXX11_ABI=0, ABI.

Geoffrey Thomas helped to confirm that devtoolset-7 does the Right
Thing here, as much as that's possible.

Its libstdc++.so is actually the following linker script:

# cat /opt/rh/devtoolset-7/root/usr/lib/gcc/x86_64-redhat-linux/7/libstdc++.so
/* GNU ld script
   Use the shared library, but some functions are only in
      the static library, so try that secondarily.  */
      OUTPUT_FORMAT(elf64-x86-64)
      INPUT ( /usr/lib64/libstdc++.so.6 -lstdc++_nonshared )

The INPUT command instructs ld to search for GCC symbols in the
system's libstdc++_s.so first; if they're present there, the resulting
binary will load them from it at runtime, and the binary will match
the ABI policy described in the PEP.  If the binary requires newer
symbols that aren't present in the system's libstdc++.so, ld will
statically link them in from its interal libstdc++_nonshared.a.  This
result will also match the ABI policy, in that it will either depend
only on the subset of symbols available in CentOS 6's default
libstdc++.so or none at all.

See the ld documentation for an explanation of INPUT:

https://sourceware.org/binutils/docs/ld/File-Commands.html

Geoffrey did point out that there's a potential issue if two C++
libraries end up with their own statically-linked implementations
std::string or std::list, whose ABIs changed with GCC 5.  If instances
of these classes are allocated in a library with pre-GCC 5 versions
and freed in another with post-GCC 5 versions something bad might
happen (at least, that's my understanding).

I'm unsure how serious this issue will be in practice but it's worth
discussion!

--
  Mark Williams
  mrw at twistedmatrix.com