[Distutils] People want CPAN :-)

[Robert Kern]

On 2009-11-11 15:59 PM, Tarek Ziadé wrote:
> On Wed, Nov 11, 2009 at 10:36 PM, Robert Kern<robert.kern at gmail.com>  wrote:
> [..]
>>
>> It does feel something like that. The build system is just one of the
>> problems with distutils' internals, in my experience. You can think of the
>> rest of distutils as a little application framework for command line
>> utilities. I think this framework simply fails to provide in very
>> fundamental ways, like the "extend commands by subclassing" design pattern.
>> That choice makes it fundamentally difficult to combine extensions together.
>> I really don't see a way to evolve away from that (and believe me, over the
>> last decade, I've tried). You just need to redesign the internals if you
>> want to get away from that. You can't get from any point A to any point B by
>> evolving in small steps that are functional (not to mention backwards
>> compatible!) all the way through.
>
> I am very surprised about this statement.
>
> What did you tried for the paste decade and failed to do ? I hear some
> complaints
> since a week, but beside's David examples I didn't read any other
> precise use cases.
>
> We're looking through the build_ext use case, and we are making some
> improvement on
> the other thread. So why not doing this in other issues ?
>
> Let's discuss your use case. And if it means adding new options to run
> arbitrary commands like
> post/pre hooks to a given command, to avoid subclassing an existing
> command, let's do it.

http://svn.scipy.org/svn/numpy/trunk/numpy/distutils/command/

All of it. Now consider that here we are also trying to play nicely with the 
setuptools extensions, and Pyrex, and David is working on integrating Cython 
support.

To get to one real specific problem, let's consider build_src. build_src is a 
new subcommand in numpy.distutils that builds C extension sources from other 
files. We use this to hook in f2py's wrapper generator and other more ad hoc 
forms of generating wrappers. When build_ext uses --inplace, we need build_src 
to use --inplace, too, because it will often output some "final" products in 
addition to intermediate wrapper sources. In order to integrate this with 
setuptools' develop command (which invokes build_ext --inplace but not build_src 
--inplace because setuptools knows nothing about numpy.distutils), we need to 
create a subclass of setuptool's develop command that will reinitialize 
build_src with the appropriate option. Then we need to conditionally place the 
develop command into the set of command classes so as not to introduce a 
setuptools dependency on those people who don't want to use it.

This is nuts. numpy.distutils shouldn't have to know anything about setuptools 
to accomplish this if the framework were properly designed. And this doesn't 
even get into the fact that many of the numpy.distutils command classes that are 
shared with setuptools are conditional subclasses and probably still buggily 
cobbled together.

-- 
Robert Kern

"I have come to believe that the whole world is an enigma, a harmless enigma
  that is made terrible by our own mad attempt to interpret it as though it had
  an underlying truth."
   -- Umberto Eco