[Distutils] Extracting distutils into setuptools

[Steve Dower]

On 26Sep2017 1930, xoviat wrote:
> This was a comment by @zooba (Steve Dower):
> 
>  > (FWIW, I think it makes *much* more sense for setuptools to fix this 
> by simply forking all of distutils and never looking back. But since we 
> don't live in that world yet, it went into distutils.)
> 
> And here is my response:
> 
>> Since you mention it, I agree with that proposal. But currently we have core developers contributing to distutils and @jaraco contributing to setuptools. @jaraco is quite competent, but I doubt that he would be able to maintain an independent fork of distutils by himself.
> 
>> In short, I think your proposal is a good one, but how can we allocate manpower?
> 
> (issue31595 on bugs.python.org <http://bugs.python.org>)
> 
> So what do others think of this? My sense of things is that people are 
> open to the idea, but there isn't a plan to make it happen.

Just to put it out there, I know this is asking a lot of Jason and the 
setuptools maintainers. I don't make the proposal lightly. But 
especially in the world of PEP 517 I would rather not have an array of 
new libraries depending on distutils for their ability to compile 
extension modules. As long as there is "core" functionality to do this, 
we won't ever get any innovation out there.

For an example of innovation, I've already published a library that can 
find VS 2015 and VS 2017 installs on Windows, generate the correct 
project (make) files and build an extension.[1] It's far more reliable 
and customisable than distutils - and I've put barely any effort into 
it! - but as long as people have to choose between a dependency and a 
core library nobody will ever get to take advantage of it. (And, as 
occurred, I'll end up porting pieces into core CPython so that we don't 
break the world by *really* deprecating distutils.)

setuptools is totally welcome in my book to simply copy the compiler 
infrastructure we already have from core and never look back. It really 
does need to be maintained separately from CPython, especially on 
Windows where we continue to get innovation in the targeted tools. I 
know it's a big ask, and it's one that I can't personally commit real 
time to (though I obviously will as much as possible), but I do think it 
is necessary for our ecosystem to not be tied to CPython release cycles.

Cheers,
Steve

[1]: The library is at https://pypi.org/project/pyfindvs/ and the docs 
do not refer to the compilation part at all. Look at 
https://github.com/zooba/pyfindvs/tree/master/pyfindvs/msbuildcompiler 
to see it.