[Distutils] setup.py build_ext --rpath behaviour

[Floris Bruynooghe]

Hello

Distutils allows you to use a handy --rpath option to the build_ext
command to add an RPATH value into the linked object.  However some of
the semantics are not great IMHO.


For the first issue some background first.  On SysV-like systems
(systems using the ELF binary format) the RPATH field was created in
the .dynamic section to allow the shared object to specify extra
locations to look for their required shared objects.  However this did
overwrite the use of the LD_LIBRARY_PATH and that was no good for
administrators, so another new field was added: RUNPATH.  This was
only used *after* LD_LIBRARY_PATH instead of before it solving the
problem.  When both fields are present the runtime linker ignores
RPATH (it is impossible to create a shared object with a RUNPATH but
no RPATH).

The Solaris linker decided to always encode RUNPATH into the shared
object whenever an RPATH is specified (using the -R option).  This is
a very sensible option most likely the correct behaviour.  The GNU
linker however decided that backwards compatibility was more important
and to make it add the RUNPATH field you have to pass
--enable-new-dtags to the linker, if you use -R or -rpath on it's own
you will only get an RPATH, no RUNPATH.

Now when using the --rpath option to the build_ext command
distutils.unixcompiler.UnixCCompiler.runtime_library_dir_option() will
use some heuristics to figure out which option to pass to compiler to
get the runpath in (i.e. "-R" or "-Wl,-R" etc).  I'm going to argue
that this needs to be extened pass in -Wl,--enable-new-dtags,-R if the
GNU linker is used so that the newer and better RUNPATH gets put into
the shared objects all the time.  (I don't yet know how to detect the
GNU linker but would like a consensus on the desired behaviour before
looking into this).

Note that this is not completely a disaster, thanks to
distutils.sysconfig.customize_compiler() adding the contents of the
LDFLAGS environment variable to the command line of the linker
invocation you can work around this currently.


The second issue with build_ext --rpath is on AIX.  Again some
background on AIX shared objects, AIX is not SysV-like and uses the
XCOFF binary format instead of ELF.  Therefore they don't have a RPATH
or RUNPATH, but they do have a think called LIBPATH which does
something similar.  The difference between XCOFF's LIBPATH and ELF's
RUNPATH is that AIX's runtime linker does not have a default search
path, hence the full search path needs to be encoded into the LIBPATH
of the shared object.

Now I would propose for build_ext --rpath to encode the LIBPATH when
used on AIX since that is the correct thing to do IMHO.

(Implementation note: since this is done by passing
-blibpath:/opt/example.com/lib:/usr/lib:/lib to the linker note that
distutils.unixccompiler.UnixCCompiler.link() would have to be changed
not to strip out /usr/lib and /lib from the runtime_library_dirs on
AIX.  And
distutils.unixccompiler.UnixCCompiler.runtime_library_dir_option()
would have to use -blibpath:... or -Wl,-blibpath as appropriate)

Again, this can currently be circumvented by using the LDFLAGS
environment variable.


Do these improvements sound sensible?  And if so should I create one
patch for each (and two bug reports) or combine them into one patch?


Regards
Floris

-- 
Debian GNU/Linux -- The Power of Freedom
www.debian.org | www.gnu.org | www.kernel.org