[Distutils] Add additional file categories for distutils, setuptools, wheel

[Paul Moore]

On 19 April 2015 at 17:41, Nick Coghlan <ncoghlan at gmail.com> wrote:
> On 19 April 2015 at 06:03, Paul Moore <p.f.moore at gmail.com> wrote:
>> As a possible compromise, how about an approach where on Linux system
>> installs (or more accurately "those install schemes that are relevant
>> to the distribution packaging issue you described") the file
>> categories are installed into dedicated directories, as described. But
>> on other installs (virtualenvs, Windows, maybe OSX) the file
>> categories map to locations within package_data, so that they can be
>> accessed via normal mechanisms like loader.get_data. Application code
>> would need some support code to locate and read the files, but that's
>> true whether we go for this proposal or an "outside of site-packages"
>> scheme. Also, some things may even be better designated as "don't
>> install" in certain schemes (man files, for example, are pretty much
>> useless on Windows).
>
> That's not a compromise, that's exactly what I want to see happen :)
>
> If it helps, one way to think of this is as a "file classification"
> system, where the packaging tools gain the ability to sort files into
> a richer set of categories, and its then up to installers to decide
> how those categories map to installation locations. For Windows,
> virtualenv, conda, nix, single file applications, etc, that's likely
> to be "self-contained application directory". For FHS, it will map to
> the on-disk categorisation expected by other tools.
>
> At the moment, because the FHS support is either hacked in, or managed
> externally, there's no way for installers to remap the installation
> targets for the "self-contained directory" use case.

OK, that's fantastic. It sounds much more reasonable when put that
way. The only big debate then is likely to be over the precise
categories used. And in particular, should we simply take an existing
Unix-centric set of categories like the autotools ones, as currently
proposed, or should we choose something more cross-platform?

In favour of Unix autotools-style classes:

1. Already exists, easy to just take what's already defined.
2. Proven utility (although possibly over-engineered, some of the
autotools categories seem pretty obscure to my admittedly untrained
eye).
3. Mapping to Unix systems (and hence distribution packaging systems)
is obvious.
4. Generally, Windows users don't care about any of this as the model
on Windows is to install everything in the application directory, so
following Unix conventions will be less controversial.

Against:

1. The categories are pretty meaningless for Windows developers, so
would tend to be ignored or used incorrectly by them.
2. Makes it too easy for Unix users to ignore or misunderstand
cross-platform issues (for example, the mandir category implies that
shipping man pages is a reasonable way of documenting your package,
but Windows users won't be able to read the manpages, and they may not
even get installed on Windows).
3. Easy to interpret as "treating Windows as a second class citizen".

Overall, I don't see enough disadvantages in there to argue against
the autotools classifications, but I would like to see the PEP include
a discussion of the portability implications of the new feature.

And there are still a load of questions waiting on a clear spec of how
installers are expected to map the categories to filesystem locations,
for all the various OS/install scheme combinations that exist. But
it's not worth speculating on those, better to wait for the spec and
then discuss the details.

Paul