[Distutils] Comments on PEP 426

[Oscar Benjamin]

On 4 September 2013 11:30, Donald Stufft <donald at stufft.io> wrote:
>
> On Sep 4, 2013, at 6:21 AM, "M.-A. Lemburg" <mal at egenix.com> wrote:
>
>> I quite like the idea of using setup.py as high level
>> interface to a package for installers to use, since that
>> avoids having to dig into the details built into the
>> setup.py code (and whether it uses setuptools, bento,
>> custom code, etc.).
>
> I like it as a temporary solution that is backwards compatible with the old
> tooling but I don't think it should be the interface going into the future.

It shouldn't be the recommended interface for new packages but it will
need to be supported at least as a backward compatibility mode for a
*long* time.

> If I recall correctly Tarek started out trying to improve distutils and ended
> up inadvertently breaking things which ended up getting his changes
> backed out and the block on changes to distutils was placed and the
> distutils2/packaging effort was started.

As I remember it, the problem Tarek had was that a __private
function/method somewhere in distutils was being used by setuptools.
Tarek changed the behaviour of the private function which resulted in
breakage for setuptools so the changes were backed out. It was then
decided that if even modifying private interfaces could not be done in
a backward-compatible way then there could basically be no changes to
distutils unless absolutely required for the purposes of building
CPython itself.

You could argue that this was perhaps something of an over-reaction.
For example the changes proposed here are to add new code/commands
rather than modify existing ones. The potential for breakage is a lot
lower in this case.

Also Tarek's problems were IIRC confounded by setuptools being
unmaintained at the time so that the onus was entirely on Tarek not to
make any change that could break anything for setuptools. At the time
it seemed to be considered that there could be no expectation that
setuptools itself could make reasonable adjustments in preparation for
or after any change in distutils. I think most projects would
understand if their setup.py breaks in a new Python version because
they were accessing private methods. The problem with setuptools was
that a lot of projects who only used the documented setuptools APIs
would have experienced breakage and then been rightly upset about the
situation.

> All of this completely skirts the fact that any change to distutils would only
> be available in 3.4+ or 3.5+ which makes it's value practically zero. It's
> not like other modules in the library where you can reasonably expect
> someone to have a backport installed if you need to use the new features.
> Setuptools has already gone through the long process of getting everyone
> to get it installed, why would we want to go through that again for a system
> that should eventually be deprecated entirely?

If there is a minimal interface that setup.py should support then I
think it's very reasonable to say that a simple distutils setup.py
script should export that interface. Specifically it should be
possible to do 'python setup.py bdist_wheel' and 'python setup.py
egg_info'. Projects with a distutils-based setup.py will be around for
many future Python versions.

Also would this be sufficient to decouple pip and setuptools (a
reasonable goal in itself)? Or does pip depend on setuptools in more
ways than the distutils monkey-patching?


Oscar