[Wheel-builders] C++ ABI v5

[Michael Sarahan]

Hi Nathaniel,

Thanks for your feedback.  It sounds like you think this won't be much of
an issue, and I really hope you're right!  This feels something like the
y2k bug to me: apocalyptic predictions, but perhaps ultimately little
impact.

The issues we have seen to a limited extent are:

- User runs some of our software, which uses our (old) C++ ABI
- In that process, our old libstdc++ now comes higher in priority than the
system one
- later, some system library goes looking in libstdc++ (ours) and barfs,
taking down the whole process.

This is really a runtime concern, unfortunately, not a build-time linking
problem.  I'm less concerned about the problem of telling people how to
compile things correctly - package builders are smart people.  I just don't
like the thought of accidental crashes when people have no idea what an ABI
is.

This is a distro/package management problem, to be sure, but I think it's
relevant for this group (and certainly for Continuum) to think about, since
we (manylinux-builders) are the packagers, and consumers of wheels that use
c++ may run into this.  I think the docker image PR I posted handles this
in a reasonable way (providing two similar GCC's, each with different
default C++ ABI settings) - and then we have to track which libstdc++ we
send to people accordingly.  This is sort of your second proposal, I
think.  We're counting on some modification to Conda in order to help
obviate the user's need to know what an ABI is, or which one they have.

If anyone wants to follow this work, or get involved, let me know.

Best,
Michael

On Wed, Mar 9, 2016 at 6:16 PM Nathaniel Smith <njs at pobox.com> wrote:

> Hi Michael,
>
> On Wed, Mar 9, 2016 at 3:49 PM, Michael Sarahan <msarahan at gmail.com>
> wrote:
> > Howdy,
> >
> > Are you all aware of the looming move to the newer C++ ABI for GCC?
> > http://developers.redhat.com/blog/2015/02/05/gcc5-and-the-c11-abi/
> >
> > This won't have humongous impact on Python as a whole, as it affects only
> > C++ packages.  However, for those packages, I think we'll see breakage on
> > new platforms.  Importantly, the upcoming Ubuntu 16.04 release will use
> that
> > newer ABI (15.10 already does, but 16.04 is LTS).  Many distros are
> ending
> > up recompiling their entire package libraries.
>
> To check if I'm understanding the problem correctly... it's not that
> C++ packages will be broken in general, right? The specific situation
> where things break is:
>
> 1) you distribute a C++ library that's built using the old ABI, AND
> 2) your C++ library has a public API/ABI that involves some of parts
> of the stdlib whose ABI changed (e.g. std::string), AND
> 3) your users running Ubuntu 15.10 (for example) want to compile their
> own code against your C++ library's API/ABI
>
> If all these things happen, then the users' builds will fail by
> default (because the users' compiler will default to using the new
> ABI, but your package is built using the old ABI), and your users will
> have to add -D_GLIBCXX_USE_CXX11_ABI=0 to their compile lines.
>
> And then AFAICT there are fundamentally two strategies one could use
> to deal with this:
> - modify your C++ library so that it exports both old- and
> new-versions of its ABI from the same .so simultaneously. This is what
> libstdc++ does, and what that blog post describes how to do. OTOH this
> requires non-trivial modifications to the package source, and it's
> unlikely that any distributor is going to try and go through and hack
> up a giant C++ library to do this (since the cases we're worried about
> here are like... Qt and LLVM, right? And now that I think about it I'm
> not even sure whether Qt's ABI is affected, since they go to some
> effort to avoid using standard library classes in their public ABI --
> QString instead of std::string, etc.)
>
> - make a build of your C++ library that just uses the new ABI, and
> figure out some way to manage the resulting hassles involved in keep
> track of the two builds -- can they be installed simultaneously? how
> do you make sure that only systems with a new libstdc++ get the new
> version of the library? or do you switch to distributing libstdc++
> yourself and push everyone to the new ABI, even if they're on an old
> distro? if you do, then can RHEL5 system gcc build against a newer
> version of libstdc++, or would this mean that you need to start
> shipping gcc as well?)
>
> > Just curious - how (and when) will the manylinux docker image tackle
> this?
> > We (Continuum) don't have a solid answer yet.  We are looking into a
> docker
> > image similar to manylinux that uses a newer GCC that can output either
> ABI:
> > https://github.com/ContinuumIO/docker-images/pull/20
>
> The manylinux image doesn't really give much consideration to the
> problem of distributing non-Python C++ libraries, at least so far... I
> guess at some point there may be wheels that just contain LLVM or Qt,
> but we're not there yet :-). In any case, the hard problems to me all
> seem to be at the distro/package management level, not at the docker
> image level -- I think once one has decided how one wants to solve
> those problems, then the docker image part should follow?
>
> -n
>
> --
> Nathaniel J. Smith -- https://vorpus.org
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/wheel-builders/attachments/20160310/88e34285/attachment-0001.html>