[Distutils] Comments on PEP 426

[Ronald Oussoren]

On 5 Sep, 2013, at 19:32, Daniel Holth <dholth at gmail.com> wrote:

> On Thu, Sep 5, 2013 at 8:33 AM, Ronald Oussoren <ronaldoussoren at mac.com> wrote:
>> 
>> On 31 Aug, 2013, at 17:56, Nick Coghlan <ncoghlan at gmail.com> wrote:
>> 
>>> 
>>> This perception is just wrong. Mac OS X is a mess due to clang vs gcc vs homebrew vs macports vs Xcode, Windows is a mess due to mingw and cygwin and platform SDKs and visual studio (Express or otherwise). Linux distros are also ridiculous, with different gcc variants, library locations and various other things, depending on flavour and version.
>> 
>> Don't forget fat binaries on OSX, those seem to cause problems as well for folks building extensions.
>> 
>>> 
>>> distutils itself is nigh impossible to work on, since it is underspecified, and the fact it was added to the standard two years before unittest still shows in its test suite. (This lack of test coverage is also a large part of the reason setuptools *isn't* quite a drop-in replacement).
>> 
>> The lack of maintainability of distutils is partly due to underspecification, but that's just as true for setuptools.  Looking in from the peanut gallery, the lack of forward movement for distutils is for a large part due to pushback at first because folks were scared to break existing code using distutils and later because it was deemed unacceptable to break setuptools.
>> 
>> This seems to lead to the sad conclusion that there will be improved package installation and distribution through wheels, metadata 2.0 and related standards but that distutils won't get to play along even though plain distutils would work just fine for a very large subset of packages.  Most of my own packages only use setuptools for its dependency resolution, and that's for the most part provided by pip in the future.
>> 
>> That's not to say that distutils is perfect, far from it, but creating a new and improved build tool that's is capable of dealing with the complexities of compiler variations will be a large project in its own right even without migrating existing projects to it.
> 
> Unfortunately there is not a malicious "modernized distutils" or
> "stdlib suitable replacement" vacuum; on the contrary great effort has
> been expended to that end. Distutils itself is not being removed or
> getting worse in an absolute sense and will continue to be about as
> suitable for packaging as it has been. A modern distutils is not
> required by but would be allowed by the cited plan. I will not do it
> because I have observed distutils2 but best of luck to someone else
> who may be willing and able to write the code.

A redesign is a good way to get burned out. I have a hard time to find the
energy to work on my opensource projects as it is :-(

> 
> Distutils is un-maintainable not due to underspecification or unit
> tests but due to its basic design. Extensions have to reinitialize or
> muck about in each other's objects to work making it difficult to
> change either the interface or the implementation without breaking
> some of the hundreds of thousands of existing sdists.

That can be cleaned up by providing better internal interfaces, and should
have a change to succeed when proceeding in small steps. The hard part will
be to find a way forward that has good enough backward compatibility.

But that's not what I'd like to see in the near future. Distutils should IMHO 
provide enough functionality to integrate into the modern packaging environment: 
support for metadata 2.0 and support for creating wheel archives.

That's all that's needed to keep distutils usable while someone tries to
design and implement a replacement for distutils or a much cleaned up version
of distutils itself.

> 
> If you can produce the modernized distutils then you may. It would be
> an acceptable plugin in the eventual pluggable build systems scheme.

I've also seen what happens when anyone tries to change distutils and
am not interested in doing any work beyond keeping OSX functioning unless
there would be buy-in before hand. That is, I'd be willing to do the work
needed to support metadata 2.0 and a bdist_wheel command in distutils
when there is a clear signal that such work wouldn't dismissed out of hand 
(as has happened in the past).

Ronald