[Distutils] Making commands extensible by default

[David Cournapeau]

Toshio Kuratomi wrote:
> Tarek Ziadé wrote:
>   
>> Hello,
>>
>> This is a side discussion but quiet important ihmo.
>>
>> == Problem ==
>>
>> Some people complained about the fact that is was hard to extend
>> Distutils commands.
>> You end up rewriting the whole command most of the time.
>>
>> So what's a command ? It's a class that is used by the distribution
>> instance when you run Distutils.
>>
>> roughly:
>>
>> cmd = Command(distribution)
>> cmd.initialize_options()
>> cmd.finalize_options()        <---  allows to check the options if
>> subcommands where run
>> cmd.run()                            <---  runs the code
>>
>> each command can define sub commands, but most of the time it's a
>> harcoded list, so you need to inherit the command
>> if you want to add a new behavior.
>>
>> == work in progress, ==
>>
>> What we want to do here is being able to define subsets in run(),
>> sharing the same options environment.
>>
>> so basically, a rough, generic run() method could be:
>>
>> def run():
>>    for func in some_funcs:
>>       func(self, options)
>>
>> If "some_funcs" could be defined by a registery with simple names,
>> anyone could provide new functions
>> and configure the registery to run a sequence of function.
>>
>> Given a command name, Distutils can get this list of function, through
>> a registery.
>> Each function could register itself into Distutils, like in what I
>> have started to work here for the manifest file:
>> see http://wiki.python.org/moin/Distutils/ManifestPluginSystem
>>
>> The ordering would be configurable through the setup.cfg file.
>>
>> Any opinion, idea for this part ?
>>
>>     
> Have you looked at paver?  It's syntax makes extension easy.
>   

Yes, paver is nice, but I don't think it solves the problem that Tarek
aimed at solving here. Let me introduce some usercases. For example, if
you want to extend say the distutils sdist command, you could do:

@task
@needs(['distutils.command.sdist'])
def mysdist():
    # Add some more stuff here to the tarball
    ...

But what if you want to alter the original sdist behavior, e.g. place
some files elsewhere ? Paver works nice if you want to extend the
existing behavior in a simple way (after doing this, do that), but
unfortunately, some things do not fit well if at all in this scheme.

I would cite two examples, related to sdist and paver:
    -
http://groups.google.com/group/paver/browse_thread/thread/581534ca19cc541e
    -
http://groups.google.com/group/paver/browse_thread/thread/e153ba3da8cc3334

There are also many things which are annoying once you start using paver
for cross-platform deployment, because of distutils commands. For
example, you may need to move some distributions files (tarball,
binaries installers, doc) somewhere else. Even though the code used to
generate the files are in python, there is currently no way to use that
code outside distutils - things as simple as platform specific names of
the tarballs, etc... are not factored out, and you have to recreate it
in your own scripts.

So the current problem with distutils commands is not just how to easily
extend them - sometimes, you need to alter them as well, hence Tarek's
suggestion for plugins/registry.

cheers,

David