[pypy-dev] Using "rpath"

[Armin Rigo]

Hi all,

Here I'm describing the best I could attempt about the problem of
binary distributions.  It's still a major mess to implement.  It may
be done some day, but basically at this point I'm looking for
contributors.

A first note: this is *only* about the Linux Binary distributions that
we provide on pypy.org/download.html.  It has no effect on the pypy
provided (later) by your particular Linux distribution.

The goal is to make it *work* in a reasonable way.  The goal is *not*
to be sure that it will automatically link with the latest version of
libssl.so.x.y.z that you happen to have installed on your system under
some non-automatically-guessable name, because the goal is not magic.
It would however let you do that manually if you care to.  (If you
want something 100% automatic, go to your own Linux distribution and
help them upgrade.)

The idea would be to have in /opt/pypy-2.0.x/bin an executable "pypy",
which dynamically links to libssl.so, libcrypto.so, libffi.so, etc.;
but to also, by default, put actual libraries in the same directory as
the executable, under the versionless names "libcrypto.so" etc., and
configure the executable in such a way that it would load these
libraries (without using LD_LIBRARY_PATH; see below).  This is for the
*work* part.  If people downloading the linux binaries want better,
they can remove the libraries and have the pypy finds them on their
system; or if their system is missing files with the exact same name,
they can instead put in /opt/pypy-2.0.x/bin some symlinks.  (This is
much better than having to stick the symlinks in /usr/lib/.)

Well, that's the goal.  It's of course not so simple.  Here is what I found out:

- First, I'm going to ignore the optional notion of "versioned symbols" here.

- In a Linux .so file, there might be an entry "SONAME" embedded in
the file, which is supposed to contain "libfoo.so.N" with a single
number N.  In the case of libcrypto/libssl it is in general not that,
of course.  It is for example "libcrypto.so.1.0.0".  The SONAME might
also be absent.

- When we compile an executable with "gcc -lfoo" or "gcc
-l:/path/libfoo.so" or just "gcc /path/libfoo.so", gcc looks up the
real libfoo.so.x.y.z and reads it.  It then puts the name of the
library inside the executable, for later dynamic linking, as can be
seen in the left column of "ldd executable".  Now the 'name' written
there is just the filename if the .so has no SONAME.  But if the .so
has an SONAME, it completely and definitely ignores the filename given
on the command-line, and puts the SONAME instead.

- Another option to gcc is "-Wl,-rpath=X" which embeds another entry
into the produced binary: RPATH, giving it the value X.  This should
be a path: the "run-time search path".  It may start with the special
string "$ORIGIN", which is resolved to "wherever the binary actually
is at run-time".

- At run-time, the entries in that table are searched first in the
RPATH if any, then in the system's default places.

On a default "gcc -lcrypto" for example, gcc finds
/usr/lib/libcrypto.so (with or without extra .x.y.z), loads its
SONAME, finds "libcrypto.so.x.y.z", and so sticks "libcrypto.so.x.y.z"
into the executable.  At run-time, the dynamic linker will thus only
look for a file "libcrypto.so.x.y.z".  I found no way to tell gcc
"ignore the SONAME", so there is no way to produce an executable that
would contain a more general name.

So the problems we're facing are: the SONAME of libssl/libcrypto are
too precise for us; and anyway the libc itself has added
incompatibilities in recent versions too.

It's stupid but it seems that the only reasonable way forward is
really to install on all buildslaves a custom chroot.  This chroot
would contain an old-enough version of the libraries like the libc.
It would also contain a *custom* compiled version of libssl/libcrypto
(and if needed libffi and all others) which does not include any
SONAME.  In this way, we can choose with gcc options what names end up
in the binary.  We can pick a general name, and package with the Linux
binaries .so's with these general names.

So, anyone up to the task?


A bientôt,

Armin.