At 03:24 PM 9/25/2008 -0700, Guido van Rossum wrote:

...

Right now there's a momentum in the community, including framework gurus, that are willing to work on a new distutils package. They are not core developers but they are really good in distribution matters. Even Phillip Eby said that starting a new distutils could be a good pick in this thread earlier.

I wasn't there. I'd like to refer to a post by Joel Spolsky about the problem with total rewrites: http://www.joelonsoftware.com/articles/fog0000000069.html

The economic factors are a bit different, here. Joel himself has previously pointed out that where Netscape failed, Mozilla won - i.e., the economics of open source can mean that it's sometimes easier to get volunteers for a new project than for fixing an old one, or at least for a project where dropping backward compatibility is allowed (e.g. Py3K). In the case of the distutils, the people who are capable of extending it are tired of doing so, and the people who have the energy and time are very unlikely to be able to work on it much without breaking something significant. Distutils is also far too flexible in some areas to be able to improve much while maintaining 100% backward compatibility -- it doesn't enforce One Obvious Way To Do It. What's more, there are very few people who've even said they like the distutils API or think it's a good fit for the application area. And, frankly, the domain knowledge embedded in the distutils are of fairly limited scope and kind: * Extension building, compile/link options and defines * Wildly-differing installation path schemes across platforms * Platform distribution formats like bdist_rpm, bdist_wininst, and bdist_msi Most other things the distutils do are either well-specified, obsolete (e.g. its internal logging and option parsing libs), or probably not worth keeping (e.g. bdist_dumb).

...

Maybe a "distutils 2" project could start outside Python, using distutils and setuptools code as legacy infrastructure, and taking back pieces of it,

Then it could be re-integrated in Python as a replacement for distutils when it is mature ?

Only if much effort and study went into the planning of this re-integration.

That's why at least some of the discussion has been around requirements gathering and PEP-writing as a first step. My own inclination is that a scalable future for distutils means an improved sdist format, the end of setup.py as an command-line interface, and community-maintained platform-specific installation tools that process source or binary distributions. Most complaints about distutils (and setuptools, for that matter) are focused on installation policy&preference issues. Making it possible and practical for a variety of tools to flourish around a standardized format (ala WSGI) seems like the way to go. Notice that the existence of eggs has already allowed buildout, virtualenv, and pyinstall to appear. But eggs don't handle installing tests or documentation, and they have to be prebuilt by platform. An improved sdist format, on the other hand, with standardized layout and metadata would address all of those issues. The tools for building this format and APIs for inspecting it would be candidates for the stdlib, in much the same way that wsgiref was - the spec is/should be stable, and those parts that are compiler/install-layout specific will need to be maintained in the stdlib anyway, for Python's own build infrastructure. In that sense, "distutils 2" would not be so much a rewrite of the distutils, as the separation of them into tools for distributing, and tools for installing, where some of the tools for installing may be community-maintained. That's the general idea, anyway. (I'm going to be away from email for a few days, so I'll probably be out of this thread 'till Tuesday.)