[Distutils] [Fwd: Re: Annoucing distribute project]

David Cournapeau david at ar.media.kyoto-u.ac.jp
Fri Sep 26 19:03:10 CEST 2008

Phillip J. Eby wrote:
> The economic factors are a bit different, here.  Joel himself has
> previously pointed out that where Netscape failed, Mozilla won - i.e.,
> the economics of open source can mean that it's sometimes easier to
> get volunteers for a new project than for fixing an old one, or at
> least for a project where dropping backward compatibility is allowed
> (e.g. Py3K).

Yes, I forgot to mention that earlier: I don't think Joel's point is
applicable to distutils. Distutils is by almost all acount a small
project (sloccount tells me Python-2.5.2/Lib/distutils is 9241 loc); one
of Joel's point is that it is more costly to write from scratch than to
refactor. But I think distutils is not "refactorable" (sorry for hurting
native English speakers): a lot of things are fundamentally broken *by
design* (option handling, running one command after the other, etc...),
and cannot be changed in a backward compatible way. It is also small
enough so that a single person can have a pretty good idea of the whole

So it maybe less costly to do something from scratch, simply because
starting from scratch would brings much more resources. For example,
some people from OS distributors complain, rightfully, about distutils:
my understanding is that they would put ressources for code which would
(optionally) make python packages "OS compliant" (FHS compliant on
linux, etc...). But not in the current state of affairs. Joel's point is
valid when you have a fixed amount of resources, e.g. a company; in open
source, the dynamics can be different [1]

> What's more, there are very few people who've even said they like the
> distutils API or think it's a good fit for the application area.  And,
> frankly, the domain knowledge embedded in the distutils are of fairly
> limited scope and kind:
> * Extension building, compile/link options and defines

This can be done without too much work: the hard work really is the
options and the corner cases. Corner cases can be solved gradually (and
there are not that many corner-cases; mostly VS stuff and some mac os X
stuff in my experience). I have done it for numscons (scons did made
this a bit easier, but I needed to read and understand almost all the
distutils and numpy.distutils code to build C code on all supported
platforms, including Visual Studio). I would be glad to help there,
that's a part I now feel relatively confident with.

> * Wildly-differing installation path schemes across platforms

I did not touch this part too much. Would it be possible to refactor
internally this part in distutils, such as we can provide an alternative
implementation, which is not backward compatible, but would be specified
from a set of requirements (FHS compliant option, etc...). By default,
setup.py users would get the current default, but if they want, they
would get a better model here.



[1] Again, talking about numpy/scipy: numpy.distutils itself was ~ 10000
loc, and it was much easier for me to start from scratch (but using
numpy.distutils knowledge) than to solve numpy.distutils problems. At
least in the case of numpy, people smarter than me tried to improve
numpy.distutils, but never did it because everytime they did something,
they broke something else. Now, it may just be that I have way more
free-time than them. But the point remain that in open source, the
dynamics can be different.

More information about the Distutils-SIG mailing list