[pypy-dev] Using "rpath"

[Davide Del Vento]

Instead of using the gcc (actually ld) option "-Wl,-rpath=X", which may 
require changes in your build/link structure you can more easily use 
patchelf which is able to edit the RPATH of an existing binary. I use it 
often an is very convenient (however I don't have the different versions 
issue)

You may also want to provide the libraries as a separate download.

Regards,
Davide Del Vento

On 05/12/2013 02:57 AM, Armin Rigo wrote:
> Hi all,
>
> Here I'm describing the best I could attempt about the problem of
> binary distributions.  It's still a major mess to implement.  It may
> be done some day, but basically at this point I'm looking for
> contributors.
>
> A first note: this is *only* about the Linux Binary distributions that
> we provide on pypy.org/download.html.  It has no effect on the pypy
> provided (later) by your particular Linux distribution.
>
> The goal is to make it *work* in a reasonable way.  The goal is *not*
> to be sure that it will automatically link with the latest version of
> libssl.so.x.y.z that you happen to have installed on your system under
> some non-automatically-guessable name, because the goal is not magic.
> It would however let you do that manually if you care to.  (If you
> want something 100% automatic, go to your own Linux distribution and
> help them upgrade.)
>
> The idea would be to have in /opt/pypy-2.0.x/bin an executable "pypy",
> which dynamically links to libssl.so, libcrypto.so, libffi.so, etc.;
> but to also, by default, put actual libraries in the same directory as
> the executable, under the versionless names "libcrypto.so" etc., and
> configure the executable in such a way that it would load these
> libraries (without using LD_LIBRARY_PATH; see below).  This is for the
> *work* part.  If people downloading the linux binaries want better,
> they can remove the libraries and have the pypy finds them on their
> system; or if their system is missing files with the exact same name,
> they can instead put in /opt/pypy-2.0.x/bin some symlinks.  (This is
> much better than having to stick the symlinks in /usr/lib/.)
>
> Well, that's the goal.  It's of course not so simple.  Here is what I found out:
>
> - First, I'm going to ignore the optional notion of "versioned symbols" here.
>
> - In a Linux .so file, there might be an entry "SONAME" embedded in
> the file, which is supposed to contain "libfoo.so.N" with a single
> number N.  In the case of libcrypto/libssl it is in general not that,
> of course.  It is for example "libcrypto.so.1.0.0".  The SONAME might
> also be absent.
>
> - When we compile an executable with "gcc -lfoo" or "gcc
> -l:/path/libfoo.so" or just "gcc /path/libfoo.so", gcc looks up the
> real libfoo.so.x.y.z and reads it.  It then puts the name of the
> library inside the executable, for later dynamic linking, as can be
> seen in the left column of "ldd executable".  Now the 'name' written
> there is just the filename if the .so has no SONAME.  But if the .so
> has an SONAME, it completely and definitely ignores the filename given
> on the command-line, and puts the SONAME instead.
>
> - Another option to gcc is "-Wl,-rpath=X" which embeds another entry
> into the produced binary: RPATH, giving it the value X.  This should
> be a path: the "run-time search path".  It may start with the special
> string "$ORIGIN", which is resolved to "wherever the binary actually
> is at run-time".
>
> - At run-time, the entries in that table are searched first in the
> RPATH if any, then in the system's default places.
>
> On a default "gcc -lcrypto" for example, gcc finds
> /usr/lib/libcrypto.so (with or without extra .x.y.z), loads its
> SONAME, finds "libcrypto.so.x.y.z", and so sticks "libcrypto.so.x.y.z"
> into the executable.  At run-time, the dynamic linker will thus only
> look for a file "libcrypto.so.x.y.z".  I found no way to tell gcc
> "ignore the SONAME", so there is no way to produce an executable that
> would contain a more general name.
>
> So the problems we're facing are: the SONAME of libssl/libcrypto are
> too precise for us; and anyway the libc itself has added
> incompatibilities in recent versions too.
>
> It's stupid but it seems that the only reasonable way forward is
> really to install on all buildslaves a custom chroot.  This chroot
> would contain an old-enough version of the libraries like the libc.
> It would also contain a *custom* compiled version of libssl/libcrypto
> (and if needed libffi and all others) which does not include any
> SONAME.  In this way, we can choose with gcc options what names end up
> in the binary.  We can pick a general name, and package with the Linux
> binaries .so's with these general names.
>
> So, anyone up to the task?
>
>
> A bientôt,
>
> Armin.
> _______________________________________________
> pypy-dev mailing list
> pypy-dev at python.org
> http://mail.python.org/mailman/listinfo/pypy-dev
>