[pypy-issue] Issue #2809: Unable to install libraries with c modules with custom rpath on certain OS (e.g. Arch) (pypy/pypy)

[Yusuke Tsutsumi]

New issue 2809: Unable to install libraries with c modules with custom rpath on certain OS (e.g. Arch)
https://bitbucket.org/pypy/pypy/issues/2809/unable-to-install-libraries-with-c-modules

Yusuke Tsutsumi:

# Steps To Reproduce #

1. create a virtualenv for pypy (bin/virtualenv-pypy)
2. run:
```
$ pip install pyzmq --global-option="build_ext" --global-option="--rpath=./Lib"
```

The output will probably look like:


```
#!python

    creating build/lib.linux-x86_64-3.5/zmq/backend/cffi
    cc -pthread -shared -Wunused-parameter -L/home/tsutsumi/Downloads/pypy3/env/lib -Wl,-rpath,/home/tsutsumi/Downloads/pypy3/env/lib build/temp.linux-x86_64-3.5/zmq/backend/cffi/__pycache__/_cffi_ext.o -R./Lib -lzmq -o build/lib.linux-x86_64-3.5/zmq/backend/cffi/_cffi_ext.pypy3-510-x86_64-linux-gnu.so
    cc: error: unrecognized command line option ‘-R’
    error: command 'cc' failed with exit status 1
    
    ----------------------------------------
Command "/home/tsutsumi/Downloads/pypy3/env/bin/pypy -u -c "import setuptools, tokenize;__file__='/tmp/pip-install-ncfdiwxd/pyzmq/setup.py';f=getattr(tokenize, 'open', open)(__file__);code=f.read().replace('\r\n', '\n');f.close();exec(compile(code, __file__, 'exec'))" build_ext --rpath=./Lib install --record /tmp/pip-record-ky03ulyu/install-record.txt --single-version-externally-managed --compile --install-headers /home/tsutsumi/Downloads/pypy3/env/include/site/python3.5/pyzmq" failed with error code 1 in /tmp/pip-install-ncfdiwxd/pyzmq/
```

# Investigation / Details #

The --rpath parameter passed into the setup.py specifies additional parameters to be passed into GCC. Those parameters are chosen by the unixcompiler.py module (which seems to be the same for both pypy and cpython):

https://github.com/python/cpython/blob/master/Lib/distutils/unixccompiler.py

Which is choosing the erroneous flags -R in lieu of -Wl for gcc.

It looks like this isn't encountered during cpython, as a result of gcc being noted as the compiler, and that forces the correct values:

https://github.com/python/cpython/blob/master/Lib/distutils/unixccompiler.py#L226

Running the sysconfig.get_config_var for CC gives nothing on pypy3:

```
$ pypy3 -c "import sysconfig; print(sysconfig.get_config_var('CC'))"
None
```

Digging a bit deeper, it looks like these variables are autopopulated by consuming the Makefile used to compile Python. In pypy3, this file does not exist.

The function responsible for populating the CC config var:

https://github.com/python/cpython/blob/master/Lib/sysconfig.py#L411

The function reading the makefile, and creating the subsequent file that is consumed to construct the config var needed to resolve to the right cc variable:

https://github.com/python/cpython/blob/master/Lib/sysconfig.py#L348

It seems like the solution could be providing a faux-makefile, or potentially patching or creating the final build file from which the variables are consumed.

I'm happy to do some work in the area and send a PR, just not sure what the best direction would be to resolve this.