[Distutils] Comments on PEP 426

[Oscar Benjamin]

On 4 September 2013 08:51, M.-A. Lemburg <mal at egenix.com> wrote:
> On 04.09.2013 09:27, Paul Moore wrote:
>> On 4 September 2013 08:13, M.-A. Lemburg <mal at egenix.com> wrote:
>>
>>> I guess that's what the suggestion is all about: avoiding
>>> reinventing the wheel, endless discussions and instead going
>>> for standard software refactoring techniques.
>>
>> To my mind the biggest issue (and again, I'm with Antoine here -
>> people keep forgetting this) is that there are no defined API specs to
>> work to. You can't implement "just the important bits" of setuptools
>> without knowing what those bits are, and what the interface to them
>> is.
>
> I don't quite follow you there. setuptools does come with
> documentation and the code is available to be read and reused.
>
> The situation is similar for distutils itself. Most of the
> details are laid out in the code. You just need to take
> a bit of time and learn the concepts - which is not all
> that hard.

An implementation is not a defined API spec even if it does come with
some documentation. Tools like pip will need to install projects whose
setup.py uses distutils, or setuptools, or monkey-patched versions of
distutils/setuptools or a reimplementation of some version of
distutils, or with a setup.py that uses neither distutils nor
setuptools. What is needed is an independent specification of the
minimal command-line interface that a setup.py should provide that is
consistent with how things work now for each of those types of
setup.py and sufficient for the needs of past, present and future
packaging tools.

There is documentation for e.g. the egg_info command:
https://pythonhosted.org/setuptools/setuptools.html#egg-info-create-egg-metadata-and-set-build-tags
However this is really just a description of how to use the command
rather than a specification of what expectations can be made of it by
other tools and libraries.

The problem with implementation defined specifications is that there's
no way to reasonably fix or improve the implementation. It is never
possible to say that an implementation conforms or contravenes a
standard if the implementation *is* the standard. When pip fails to
install a project X from PyPI it is not possible to say which of X or
pip (or distutils/setuptools if used) is buggy since there is no
explicitly documented required behaviour anywhere apart from the
general expectation that 'pip install X' should work.

As a case in point 'pip install bento' does not work (fails to find
the egg info directory). I haven't discovered the reason for this yet
but I wouldn't be surprised if the reason is that bento's setup.py
differs in its behaviour in some way that isn't specified in any API
docs. If the answer is that the bento authors should read the whole of
the setuptools codebase and ensure that what they produce is exactly
the same then they may as well use setuptools and there's basically no
way for anyone to distribute sdists that don't use
distutils/setuptools.

If the expected minimal behaviour of setup.py including the relevant
parts that currently *need* to come from setuptools (i.e. the reasons
for pip to monkeypatch distutils with setuptools) were independently
specified in a PEP then those features could be incorporated into
future versions of distutils without propagating the
implementation-defined problems of the past. It would be possible for
pip and other tools to make assumptions in a backward- and
forward-compatible way and to fix bugs in all tools and libraries
since it would be clear what is a bug and what is not.


Oscar