[Distutils] People want CPAN :-)

Tarek Ziadé ziade.tarek at gmail.com
Tue Nov 10 19:17:11 CET 2009

On Tue, Nov 10, 2009 at 2:31 AM, David Cournapeau
<david at ar.media.kyoto-u.ac.jp> wrote:
> If you want to support a new source/target (say assembler), you need to
> deal at this stage (or use hacks to deal with the new  file extension
> you want to deal with, and remove it later...). So you need to copy
> build_ext, you cannot extend it.
> If you look at msvccompiler vs msvc9compiler, both compile methods are
> almost the same and copied from each other.

So here's a refactoring proposal :

- build_ext doesn't handle a compiler instance anymore (therefore it
doesn't have a "compile_type" anymore)

- instead, the Extension class handles this compile type

- right now we have a dict in ccompiler (compiler_class) that has all
the compiler classes.
  let's make it extensible through a registery using distutils.cfg
(like the commands)

so: when build_ext is run, it compiles the Extensions by using this
registery of compilers, and the type of compiler contained in each
extension. It keeps instances in a pool to reuse them if needed/

> I have given very explicit examples in this discussion - I have written
> them on the wiki last time it was discussed as requested:
> http://wiki.python.org/moin/Distutils/PluginSystem
> I don't think it is accurate to summarize my critic as a vague "do
> imperative operations".

Sorry about that, I lost track of this.

>> Last, I am not sure why you want only backward-compatible changes in distutils.
>> There's no plan to keep backward-compatibility if breaking it makes
>> DIstutils better. We will have pending deprecation warnings, that's
>> all.
> What is the intended policy: deprecate something in v1 and break it in
> v1+1 ? I am not sure this works well when you need to correct deep
> design issues which impact a lot of code (and this will be required).
> For example, I would prefer not changing numpy.distutils several times
> just to make a few API cleaner.

You just don't replace a stdlib package with its v2 like that, there
is a deprecation process to have.

Distutils is not living only through setup.py. It has some public APIs
imported and used by people.

While some deep changes are required, if setting up a deprecation is
possible, I will do it.

> I guess I don't see the point of breaking things while keeping the
> current distutils codebase. I would rather do the exact contrary: throw
> away the codebase, but keep current setup.py working through a
> conversion script. Things like sandboxing pypi and other things become
> much easier that way.


I'm hearing this sometimes. That's what I thought too when I started
to work with Distutils
a year and a half ago. And I couldn't understand Guido when he was saying that
redoing it from scratch would be a mistake. Now I agree with him.

There's a *lot* of knowledge in Distutils. Throwing it away means
entering a new cycle that will last
for several years. Small refactoring can be done today, as long as the
code is tested.

While the code is not yet as modern as other modules in the stdlib, I
invite you to compare Python 2.4's
distutils with the current trunk, and realize the amount of work I've
been doing in the past year.

My goal was to have a good coverage before starting any big
refactoring, and to fix the old pending bugs

2 years ago, the code was barely tested (the coverage was 18%) and now
it's well tested (80%+ depending on the platform). It's now almost
fully PEP 8, and I am now more confident when I start any refactoring.

I understand the pain with build_ext. let's work it out.


More information about the Distutils-SIG mailing list