[Distutils] Improving distutils vs redesigning it (was people want CPAN)

[Glyph Lefkowitz]

On Nov 11, 2009, at 10:04 PM, Robert Kern wrote:

> In our considered opinion, piecemeal changes probably aren't going to solve the significant problems that we face. At best, they simply aren't going to help; we wouldn't be able to use the new features until we can drop support for Python 2.6.

While I can understand your frustration, it's important to step back and think about whether your problems are really impossible to solve.  In particular, I take issue with this idea that you can't work on stuff that's distributed with Python but depend on newer versions.  Isn't the whole point of much of setuptools' complexity supposed to be the fact that you can have side-by-side multi-version installations?  Even assuming that this functionality doesn't work at _all_, who is to say that you can't ask users to upgrade distutils?  or do a --prefix installation of distutils into a different directory?  or have a build-time option that installs the 'distutils' package as 'distutils_plus_plus' and rewrites things as necessary?  or implement an alternative to require() which *does* work? or, assuming require() works for some cases but not yours, adapt it to your needs?

There are probably a dozen other ways that you *could* work on distutils and benefit more immediately from your efforts than the next Python release.  To think otherwise is a simply a failure of imagination.  Now, if you think it's *too hard* to do that, it might be interesting to hear why you think that, and what exactly the effort would be; a nebulous assertion that it's just too hard and we should throw our hands up (while I can definitely understand the impulse to make such an assertion) serves only to discourage everyone.

The fact that a package is in the standard library is not a death sentence.  Releases can be made separately.  Heck, if you are doing good work on trunk but the release cycles are taking too long, quite frequently distributors will make packages out of your code at some revision of trunk rather than a release.  I maintained software for _years_ that required a more recent version of pysqlite bindings than were available in the standard library's 'sqlite3' module.  The 'pysqlite2' project is alive and well, and we didn't have any significant problems.

Now, as Tarek suggests, it would be more worthwhile to discuss the *specifics* of the problems that you assert require blowing up the world, as more detailed understanding of those specifics will allow both people who want rewrites *and* people who want incremental improvements to proceed with better-informed.  Any language environment's package/distribute/build/install/run pipeline is complicated enough that one can have a lot of productive discussion just nailing down exactly what is wrong with it, before even talking about solutions, and Python is no exception.