[Distutils] People want CPAN :-)

[David Cournapeau]

Tarek Ziadé wrote:
>
> What is a registery of extension exactly ? Distutils let you register your own
> commands, you can use through the CLI.
>
> Can you provide more details ?

Sure. Right now, if you want to deal with a new source or a new target,
you need to create a new command or override one. For example, cython
has distutils extension which subclass build_ext, we in numpy do the
same. This causes several issues:
    - cython build_ext subclass distutils command, we do the same. How
to use both at the same time ? That's one typical example of the
subclassing issue Ian Bicking mentioned,
    - in the case of compiled extensions, the basic structure is to
build all object files for each extension from the compiler class
(compile method). The compile method structure is:

for obj in objects:
     if src == "bla":
         do_bla()
     elif src == "blabla":
         do_blabla()
     ...
     else:
          raise some exception for unrecognied extension.

If you want to support a new source/target (say assembler), you need to
deal at this stage (or use hacks to deal with the new  file extension
you want to deal with, and remove it later...). So you need to copy
build_ext, you cannot extend it.

If you look at msvccompiler vs msvc9compiler, both compile methods are
almost the same and copied from each other.

Now, if instead you have a dictionary {src_extension: callable}, you
would be able to add a new tool and extend existing code without
touching or copying anything. You could have syntactic sugar to have
something like:

@extension(".asm")
def assemble(....):
    do_assembler_stuff()

All the assembler-related thing would be in a separate module, and you
would only need to register for the tool explicitly when you need it
(every tool could be the same, and current distutils tools would then be
registered automatically).

> I am in for making Distutils evolve, but I need very precise real
> world use cases
> not saying that Distutils shouldn't do imperative operations).

I have given very explicit examples in this discussion - I have written
them on the wiki last time it was discussed as requested:

http://wiki.python.org/moin/Distutils/PluginSystem

I don't think it is accurate to summarize my critic as a vague "do
imperative operations".

>
> Last, I am not sure why you want only backward-compatible changes in distutils.
>
> There's no plan to keep backward-compatibility if breaking it makes
> DIstutils better. We will have pending deprecation warnings, that's
> all.

What is the intended policy: deprecate something in v1 and break it in
v1+1 ? I am not sure this works well when you need to correct deep
design issues which impact a lot of code (and this will be required).
For example, I would prefer not changing numpy.distutils several times
just to make a few API cleaner.

I guess I don't see the point of breaking things while keeping the
current distutils codebase. I would rather do the exact contrary: throw
away the codebase, but keep current setup.py working through a
conversion script. Things like sandboxing pypi and other things become
much easier that way.

cheers,

David