[Distutils] Extracting distutils into setuptools

[Steve Dower]

On 29Sep2017 1306, Donald Stufft wrote:
> There are bits of distutils that are super useful (Steve indicated the 
> compiler support as one of them), and I think a better path forward is 
> to do like we’ve done with the packaging library. Make a compiler 
> package that can handle those bits and recommend that projects that want 
> to handle compilation just reuse them (of course not mandate it!). Then 
> we can focus first on creating a good API in that package without 
> inheriting the problems of distutils, and then we can maybe work on 
> bundling that inside of distutils and turn the distutils API into a shim 
> over that (or just leaving it alone).

I'm happy enough with this approach, my only problem with it is that I 
don't want to be maintaining two versions (the new one that we want 
people to change to, and the old one so that people keep working with 
new versions of Python without having to change to the new one).

If we don't make a very clear statement that "distutils is no longer 
being maintained, it probably won't build valid extensions on 
3.[7/8/whatever], and we mean it this time" then it doesn't count. I 
insist on at least removing the distutils tests from the test suite, 
since as long as those need to pass I can never stop maintaining the 
stdlib version. And that puts this in the realm of python-dev, which 
means a PEP to formally deprecate distutils (that I'm happy to 
co-author) and recommend an alternative, as well as the alternative path 
needed to build the core extension modules on non-Windows platforms.

Whatever breaks setuptools's dependency on distutils helps drastically 
with this, as then we can recommend setuptools as the alternative path. 
Perhaps we can extract distutils into an installable package that will 
shadow the stdlib one, and setuptools can continue monkey patching while 
also taking a build/install-time dependency on it.

I honestly don't know what the best approach here is, which is why I 
raise the question. I'm fairly convinced that the best approach is *not* 
to have a whole lot of PEP 517 build tools that depend on distutils for 
core functionality.

Cheers,
Steve