[Distutils] Fwd: add install_paths.json to .egg-info and .dist-info directory

[Daniel Holth]

Cross-posting to distutils-sig:

As a prerequisite to adding useful finer-grained installations to
setuptools & wheel, I propose adding install_paths.json as a metadata
file within the egg-info and dist-info directories, initially
containing only the current paths as a key - value mappping, for
example it might contain the following keys:

base
data
headers
lib
libbase
platbase
platlib
purelib
scripts
userbase (~/.local)
usersite (~/.local/lib/python...)

These are the existing paths present as properties on the
distutils/setuptools install command. Later, when we add a finer
grained install scheme, those paths would also land in the new
install_paths.json file.

In setuptools (non-wheel installations from setup.py) it's very easy
to add an additional "egg_info.writers" entry point to create this
file. Wheel installers would need to be modified to generate this
file.

Recording the existing paths would solve a current problem with
customized installs e.g. "python setup.py install
--install-lib=/custom/path" ; the custom lib path is not recorded
anywhere, making it difficult to locate the library.

In C, libraries and headers are not really installed to standard
locations. There are conventions that differ between operating systems
and distributions. That is why the pkg-config command exists, which
reads a similar key : path mapping from files in /usr/share/pkgconfig/
or from PKG_CONFIG_PATH. This mapping would serve a similar purpose
for Python distributions that might need to locate more of their
installed files besides just those installed on PYTHONPATH.


Straw man API to use the recorded paths:

pkg_resources.resource_filename(pkg_resources.Requirement.parse('dist_name'),
'$lib/library.so')

The pkg_resources API could be extended to support interpolating these
paths. You would need to pass a Requirement, not an importable name
from sys.modules, to locate a particular .egg-info or .dist-info.
Under the hood pkg_resources would use
Distribution._get_metadata('_install_paths.json') to read the install
scheme used for that particular distribution.