Fwd: add install_paths.json to .egg-info and .dist-info directory
Cross-posting to distutils-sig: As a prerequisite to adding useful finer-grained installations to setuptools & wheel, I propose adding install_paths.json as a metadata file within the egg-info and dist-info directories, initially containing only the current paths as a key - value mappping, for example it might contain the following keys: base data headers lib libbase platbase platlib purelib scripts userbase (~/.local) usersite (~/.local/lib/python...) These are the existing paths present as properties on the distutils/setuptools install command. Later, when we add a finer grained install scheme, those paths would also land in the new install_paths.json file. In setuptools (non-wheel installations from setup.py) it's very easy to add an additional "egg_info.writers" entry point to create this file. Wheel installers would need to be modified to generate this file. Recording the existing paths would solve a current problem with customized installs e.g. "python setup.py install --install-lib=/custom/path" ; the custom lib path is not recorded anywhere, making it difficult to locate the library. In C, libraries and headers are not really installed to standard locations. There are conventions that differ between operating systems and distributions. That is why the pkg-config command exists, which reads a similar key : path mapping from files in /usr/share/pkgconfig/ or from PKG_CONFIG_PATH. This mapping would serve a similar purpose for Python distributions that might need to locate more of their installed files besides just those installed on PYTHONPATH. Straw man API to use the recorded paths: pkg_resources.resource_filename(pkg_resources.Requirement.parse('dist_name'), '$lib/library.so') The pkg_resources API could be extended to support interpolating these paths. You would need to pass a Requirement, not an importable name from sys.modules, to locate a particular .egg-info or .dist-info. Under the hood pkg_resources would use Distribution._get_metadata('_install_paths.json') to read the install scheme used for that particular distribution.
On 13 May 2015 at 00:44, Daniel Holth <dholth@gmail.com> wrote:
Cross-posting to distutils-sig:
As a prerequisite to adding useful finer-grained installations to setuptools & wheel, I propose adding install_paths.json as a metadata file within the egg-info and dist-info directories, initially containing only the current paths as a key - value mappping, for example it might contain the following keys:
base data headers lib libbase platbase platlib purelib scripts userbase (~/.local) usersite (~/.local/lib/python...)
These are the existing paths present as properties on the distutils/setuptools install command. Later, when we add a finer grained install scheme, those paths would also land in the new install_paths.json file.
+1, this sounds like a good incremental step forward to me. I'm not sure how best to document it, though. Perhaps have this initial increment as an implementation detail, and then formalise it in the next version of the wheel spec. Cheers, Nick. -- Nick Coghlan | ncoghlan@gmail.com | Brisbane, Australia
From: Nick Coghlan <ncoghlan@gmail.com> On 13 May 2015 at 00:44, Daniel Holth <dholth@gmail.com> wrote:
As a prerequisite to adding useful finer-grained installations to setuptools & wheel, I propose adding install_paths.json as a metadata
+1, this sounds like a good incremental step forward to me. I'm not
sure how best to document it, though. Perhaps have this initial> increment as an implementation detail, and then formalise it in the next version of the wheel spec.
Sounds like a reasonable idea to me, too. But IIUC this file will be written by an installer, but not necessarily present in the wheel - is that right? If not present in the wheel (or perhaps anyway) ISTM it makes sense to specify it in PEP 426 because it's part of the installation metadata. Regards, Vinay Sajip
No, PEP 426 needs to be static metadata that is not modified by the installer. The new json file would make more sense in (a successor to) PEP 376 "Database of Installed Python Distributions". In wheel it would would need to be handled as a processing step similar to writing console_scripts, or doing non-console_scripts #! rewriting, or updating MANIFEST. Too bad the wheel post-install is not as pluggable as setuptools egg-info writers. On May 12, 2015 5:36 PM, "Vinay Sajip" <vinay_sajip@yahoo.co.uk> wrote:
From: Nick Coghlan <ncoghlan@gmail.com> On 13 May 2015 at 00:44, Daniel Holth <dholth@gmail.com> wrote:
As a prerequisite to adding useful finer-grained installations to setuptools & wheel, I propose adding install_paths.json as a metadata
+1, this sounds like a good incremental step forward to me. I'm not
sure how best to document it, though. Perhaps have this initial> increment as an implementation detail, and then formalise it in the next version of the wheel spec.
Sounds like a reasonable idea to me, too. But IIUC this file will be written by an installer, but not necessarily present in the wheel - is that right? If not present in the wheel (or perhaps anyway) ISTM it makes sense to specify it in PEP 426 because it's part of the installation metadata.
Regards,
Vinay Sajip
From: Daniel Holth <dholth@gmail.com>
No, PEP 426 needs to be static metadata that is not modified by the installer. The new json file would make more sense in (a successor to) PEP 376 "Database of Installed Python Distributions".
Agreed. That's what I meant :-) Regards, Vinay Sajip
participants (3)
-
Daniel Holth -
Nick Coghlan -
Vinay Sajip