[Distutils] Add additional file categories for distutils, setuptools, wheel

[David Cournapeau]

Thanks for pushing this Daniel.

I think we should wait for a bit before making this proposal official. Just
after I made my PR against wheel to implement this, I had some discussions
with Nathaniel Smith from numpy, where he remarked we may want to support
better "everything in site-packages" model.

At the lowest level, the supported schema supports both usecases. We know
that because distributions like NixOS support the "one directory per
package" model even though most packages they package use autotools scheme.

But we may want to add higher level support at the same time as this new
scheme to avoid people coming up with their own custom solutions. One idea
that was thrown out was enabling a pkg-config-like mechanism to separate
where files are from how to find information for building things. That
would allow inside site-packages and outside site-packages schemes to work
seamlessly.

I can work on a "paperman" implementation of this on top of the "wheel"
installer for the end of this week. I think that would both alleviate some
concerns for people interested in "everything in package directory", and
make the discussion more focused.

On Sat, Apr 18, 2015 at 11:24 PM, Daniel Holth <dholth at gmail.com> wrote:

> I am working on a minor update to the wheel format to add more
> categories under which files can be installed. Constructive comments
> welcome.
>
> Distutills, setuptools, wheel currently have the best support for
> installing files relative to ('purelib', 'platlib', 'headers',
> 'scripts', 'data') with 'data' usually meaning '/' or the root of the
> virtualenv.  In practice only exactly one of the 'purelib' or
> 'platlib' locations (which are usually mapped to the same directory on
> disk), and sometimes 'scripts' is used for any given package, and
> Python packages have no good way of loading any of their package files
> at run time if they are installed into any location not relative to
> sys.path.
>
> This works fairly well for Python libraries, but anyone packaging
> applications for a Linux distribution is required to follow the
> filesystem hierarchy standard or FHS.
>
>
> http://docs.fedoraproject.org/en-US/Fedora/14/html/Storage_Administration_Guide/s1-filesystem-fhs.html
> ,
> http://www.debian.org/releases/etch/amd64/apcs02.html,
> http://www.pathname.com/fhs/
>
> It would help Linux distribution package maintainers and Python
> application (not library) developers if wheel had better support for
> installing files into the FHS, but it would help everyone else who
> wanted to generate cross-platform packages if the FHS was not
> hardcoded as paths in the data/ category. To that end we should come
> up with some additional categories that map, or do not map, to the FHS
> based on the end user's platform.
>
> Distutils2 had the concept of resource categories:
> http://alexis.notmyidea.org/distutils2/setupcfg.html#resources
>
> """
> Default categories are:
>
> * config
> * appdata
> * appdata.arch
> * appdata.persistent
> * appdata.disposable
> * help
> * icon
> * scripts
> * doc
> * info
> * man
> """
>
> GNU has directory variables:
> https://www.gnu.org/prep/standards/html_node/Directory-Variables.html
> , for example prefix, exec_prefix, bindir, sbindir.
>
> Bento has a list based on the GNU paths, and allows new paths to be
> defined:
>
> prefix: install architecture-independent files
> eprefix: install architecture-dependent files
> bindir: user executables
> sbindir: system admin executables
> libexecdir: program executables
> sysconfdir: read-only single-machine data
> sharedstatedir: modifiable architecture-independent data
> localstatedir: modifiable single-machine data
> libdir: object code libraries
> includedir: C header files
> oldincludedir: C header files for non-gcc
> datarootdir: read-only arch.-independent data root
> datadir: read-only architecture-independent data
> infodir: info documentation
> localedir: locale-dependent data
> mandir: man documentation
> docdir: documentation root
> htmldir: html documentation
> dvidir: dvi documentation
> pdfdir: pdf documentation
> psdir: ps documentation
>
> I would like to add Bento's list to wheel and to setuptools. We would
> fix the disused setup.py setup(data_files = ) argument so that it
> could be used with $ substitution, or provide a utility function that
> would expand them in setup.py itself:
>
> data_files = { '$libdir/mylib' : [ 'some_library_file'],
> '$datadir/mydata' : ['some_data_file']}
>
> We would provide a default configuration file that mapped the
> categories to their installed locations.
>
> We would store the actual paths used at install time, so a package
> could look for files relative to the $datadir used when it was
> installed.
>
> Then it would be easier to distribute Python programs that need to
> install some files to paths that are not relative to the site-packages
> directory.
>
> - Daniel Holth
> _______________________________________________
> Distutils-SIG maillist  -  Distutils-SIG at python.org
> https://mail.python.org/mailman/listinfo/distutils-sig
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/distutils-sig/attachments/20150419/e56fbab0/attachment.html>