[Distutils] Comments on PEP 426

[M.-A. Lemburg]

On 04.09.2013 11:49, Oscar Benjamin wrote:
> On 4 September 2013 08:51, M.-A. Lemburg <mal at egenix.com> wrote:
>> On 04.09.2013 09:27, Paul Moore wrote:
>>> On 4 September 2013 08:13, M.-A. Lemburg <mal at egenix.com> wrote:
>>>
>>>> I guess that's what the suggestion is all about: avoiding
>>>> reinventing the wheel, endless discussions and instead going
>>>> for standard software refactoring techniques.
>>>
>>> To my mind the biggest issue (and again, I'm with Antoine here -
>>> people keep forgetting this) is that there are no defined API specs to
>>> work to. You can't implement "just the important bits" of setuptools
>>> without knowing what those bits are, and what the interface to them
>>> is.
>>
>> I don't quite follow you there. setuptools does come with
>> documentation and the code is available to be read and reused.
>>
>> The situation is similar for distutils itself. Most of the
>> details are laid out in the code. You just need to take
>> a bit of time and learn the concepts - which is not all
>> that hard.
> 
> An implementation is not a defined API spec even if it does come with
> some documentation. Tools like pip will need to install projects whose
> setup.py uses distutils, or setuptools, or monkey-patched versions of
> distutils/setuptools or a reimplementation of some version of
> distutils, or with a setup.py that uses neither distutils nor
> setuptools. What is needed is an independent specification of the
> minimal command-line interface that a setup.py should provide that is
> consistent with how things work now for each of those types of
> setup.py and sufficient for the needs of past, present and future
> packaging tools.
> 
> There is documentation for e.g. the egg_info command:
> https://pythonhosted.org/setuptools/setuptools.html#egg-info-create-egg-metadata-and-set-build-tags
> However this is really just a description of how to use the command
> rather than a specification of what expectations can be made of it by
> other tools and libraries.
> 
> The problem with implementation defined specifications is that there's
> no way to reasonably fix or improve the implementation. It is never
> possible to say that an implementation conforms or contravenes a
> standard if the implementation *is* the standard. When pip fails to
> install a project X from PyPI it is not possible to say which of X or
> pip (or distutils/setuptools if used) is buggy since there is no
> explicitly documented required behaviour anywhere apart from the
> general expectation that 'pip install X' should work.
> 
> As a case in point 'pip install bento' does not work (fails to find
> the egg info directory). I haven't discovered the reason for this yet
> but I wouldn't be surprised if the reason is that bento's setup.py
> differs in its behaviour in some way that isn't specified in any API
> docs. If the answer is that the bento authors should read the whole of
> the setuptools codebase and ensure that what they produce is exactly
> the same then they may as well use setuptools and there's basically no
> way for anyone to distribute sdists that don't use
> distutils/setuptools.
> 
> If the expected minimal behaviour of setup.py including the relevant
> parts that currently *need* to come from setuptools (i.e. the reasons
> for pip to monkeypatch distutils with setuptools) were independently
> specified in a PEP then those features could be incorporated into
> future versions of distutils without propagating the
> implementation-defined problems of the past. It would be possible for
> pip and other tools to make assumptions in a backward- and
> forward-compatible way and to fix bugs in all tools and libraries
> since it would be clear what is a bug and what is not.

It would certainly be a good idea to document a minimal
standard setup.py command line interface as PEP.

I quite like the idea of using setup.py as high level
interface to a package for installers to use, since that
avoids having to dig into the details built into the
setup.py code (and whether it uses setuptools, bento,
custom code, etc.).

Package authors could then make sure that they adhere to this
interface spec to avoid breakage such as the one you are
describing.

Could the pip developer perhaps come up with such a minimal
set of requirements they have for the setup.py interface ?

That said, it does, of course, make sense to also document the
distutils internals some more. I was just pointing out that
in order to work on or extend distutils you currently have
to look at the code, since there is only very little developer
documentation available for extending distutils.

-- 
Marc-Andre Lemburg
eGenix.com

Professional Python Services directly from the Source  (#1, Sep 04 2013)
>>> Python Projects, Consulting and Support ...   http://www.egenix.com/
>>> mxODBC.Zope/Plone.Database.Adapter ...       http://zope.egenix.com/
>>> mxODBC, mxDateTime, mxTextTools ...        http://python.egenix.com/
________________________________________________________________________

::::: Try our mxODBC.Connect Python Database Interface for free ! ::::::

   eGenix.com Software, Skills and Services GmbH  Pastor-Loeh-Str.48
    D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg
           Registered at Amtsgericht Duesseldorf: HRB 46611
               http://www.egenix.com/company/contact/