[Distutils] What is the official position on distutils?

[Nick Coghlan]

On 29 August 2016 at 03:05, Brett Cannon <brett at python.org> wrote:
> The discussion of Sylvain's proposed changes to distutils suggests that
> there isn't a clear-cut agreement or position of this SIG -- and thus Python
> -- on changes to distutils, its future, etc. Is there an official position
> I'm not aware of? If not, could we get one so we know how to handle any more
> proposed changes to distutils?

I agree it's not clear, and we should be more explicit about it.

> For me personally (and to start this conversation if necessary), I view
> distutils as the package in the stdlib that handles building extension
> modules in the stdlib for Python. That view means that if something doesn't
> service that explicit goal then it shouldn't go into distutils.

I think that's a reasonable place to draw the line for larger features
and major refactorings, especially with pip injecting setuptools for
"pip wheel" and "pip install", even if the setup.py itself only
imports distutils.

However, in cases where distutils *does* provide a particular piece of
functionality (e.g. the recent switch to emitting sdists as "tar.gz"
by default, regardless of platform), then we should maintain a
"distutils first" policy, make the change to distutils for future
versions of Python, and then backport it to earlier versions via
setuptools.

The grey area is cases like the one Sylvain is proposing, where it's a
straightforward API addition to distutils with no backwards
compatibility implications, and where we (as in distutils-sig/PyPA)
want to encourage people to write build scripts a certain way (in this
case, failing with a useful error message when particular compiler
features aren't available, rather than confronting end users with a
cryptic C/C++ compiler error).

For these cases, I think the value of including the change in
distutils isn't so much a practical software development one, as it is
an advertising opportunity: the What's New for 3.6 can not only
mention the addition and recommendation for use, it can point people
to setuptools as a version independent way of accessing the same
capability. There are still a *lot* of people out there for whom the
standard library *is* Python, and the feature releases are one of our
best opportunities to reach those folks with notifications that the
world has changed around them, and they may need to adapt their
development practices accordingly.

The backwards compatibility situation for CPython feature releases
itself also isn't as dire as it used to be - there are only a few
tightly coupled distutils extensions (with setuptools and
numpy.distutils being the most high profile ones), and it's rare these
days for us to get through a beta period without finding and resolving
any problems that come up (it's also worth remembering that the
original major problem wasn't with a feature release, it was with a
setuptools-breaking internal refactoring that landed in a maintenance
release, so there was no beta period to find and fix compatibility
problems).

Cheers,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia