[Distutils] What is the official position on distutils?

[Paul Moore]

On 28 August 2016 at 18:05, Brett Cannon <brett at python.org> wrote:
> The discussion of Sylvain's proposed changes to distutils suggests that
> there isn't a clear-cut agreement or position of this SIG -- and thus Python
> -- on changes to distutils, its future, etc. Is there an official position
> I'm not aware of? If not, could we get one so we know how to handle any more
> proposed changes to distutils?
>
> For me personally (and to start this conversation if necessary), I view
> distutils as the package in the stdlib that handles building extension
> modules in the stdlib for Python. That view means that if something doesn't
> service that explicit goal then it shouldn't go into distutils. Now that we
> are moving down the road towards making the build step configurable I'm fine
> with saying that distutils is not expected to work for people other than
> Python itself and others can use setuptools, enscons, or other projects
> while we continue to improve the situation to where the build system is just
> something you specify in pyproject.toml to generate your wheel. IOW
> distutils is only exposed publicly because Python doesn't hide anything, but
> making it useful for the general case of direct usage by people is a
> non-goal of the package.

FWIW, my view is:

* distutils handles building of Python packages, both in the stdlib and outside
* in practice, setuptools is almost always used with distutils, and
any proposed change to distutils could be made to setuptools just as
easily
* patches for distutils typically suffer from a lack of maintainers
motivated to work on them
* there are a few developers with a strong investment in distutils,
who have code that is deeply entwined with the internals of distutils,
and changes to distutils come with a high risk of causing issues for
those people
* these developers *don't* use setuptools, so changes in setuptools
don't tend to suffer from the same level of backward compatibility
constraints
* implementing changes in setuptools is *far* easier than doing so in
distutils, for the above reasons
* changes in setuptools can be used with any version of Python,
distutils changes are only available to people with no need to support
anything but the latest version (which in practice is a vanishingly
small group)
* as a result, there would have to be an *extremely* compelling reason
to make a change in distutils rather than in setuptools - sufficient
to justify the additional risk, the extra developer effort needed, and
the fact that any such change is only going to benefit users of newer
versions of Python

The reality is simply that distutils is in a somewhat unhappy place,
with too few interested maintainers and serious compatibility problems
to deal with, but that's not exactly news to anyone. I don't think
that a formal statement will help much. Past history has shown that
making pronouncements like "distutils is frozen" has not helped anyone
- so we're just trying to work with the status quo.

With regard to Sylvain's proposed changes, I haven't really seen any
arguments justifying making the change in distutils rather than in
setuptools - the last suggestion I saw was to make the change in
*both* places, but I don't see any particular reason (beyond abstract
"it makes more sense to have the change in distutils" arguments) for
not just targeting setuptools. I may be missing something (as I'm not
personally in a position where I'd be comfortable reviewing the
change) but it seems to me that it's a bit of a non-issue - just
implement in setuptools and move on. The rhetoric about "this means
that distutils is dead" doesn't feel particularly helpful.

Paul