[Distutils] [Fwd: Re: Annoucing distribute project]

David Cournapeau david at ar.media.kyoto-u.ac.jp
Fri Sep 26 04:53:00 CEST 2008


Guido van Rossum wrote:
> I don't think anyone loves distutils, or even likes it. Unfortunately
> replacing it will be very tough, because distutils has arcane but
> important knowledge built into it about many platforms, and a new
> from-scratch system is unlikely to reach that level of coverage for
> years.

Well, yes and no. Yes, there is some arcane knowledge in distutils, but
the information is already there. Incidentally, a problem with distutils
is that it is very difficult to reuse this knowlege in a programmatic
way. For example, my first 'real' contact with distutils was to try
extending it to build ctypes extensions (that is pure C dynamically
loaded libraries loadable from ctypes). It should have been easy, since
from a compile/build POV, that's the same than a C python extension.
Unfortunately, the options are buried into the code, and worse, their
location depend on the used platform/compiler (for example,
distutils.sysconfig is not usable with Visual Studio). At the end, I
gave up, doing it in a cross-platform way without copying all the
knowledge from distutils was too difficult (for me at least).

I was fed up with distutils complexity in numpy/scipy, so I started
using scons within distutils to build all compiled code in numpy/scipy.
I spent more time dealing with distutils idiosyncraties than writing the
core new system. I am certainly not advocating using scons, but the fact
is that in a couple of months, I, a not that knowledgable person about
python and cross-platforms issues at that point, manage to build numpy
and scipy without using distutils *at all* for C/C++/Fortran extensions.
And numpy is certainly far from trivial to build (numpy distutils
extensions is ~ 10 kloc), and deal with quite a variety of compilers and
platforms.

Maybe there is something to be learnt here: we could have a new small
subproject to deal with building extensions, and plug it into distutils
(e.g. have an extremely simplified scons; scons is overkill for the vast
majority of python packages, and certainly not integrable in python
core; several small projects in that space already exist). Something
like rake; the arcane knowledge (for compiled extensions at least) could
be moved gradually from distutils to this submodule. Would that fit into
your vision or not ?

cheers,

David


More information about the Distutils-SIG mailing list