[Distutils] Package install failures in 2.6.3 - setuptools vs Distribute

[K. Richard Pixley]

Alex Grönholm wrote:
> There is a lack of consensus regarding how exactly they should work. 
> If we are having this much trouble deciding how a third party tool 
> should work, it is certainly not going to be merged into distutils 
> until those issues have been resolved. Distutils is what houses (or 
> should) the parts we all agree on. That said, I think that plenty of 
> setuptools/distribute functionality should be moved to distutils 
> (after the code has been cleaned up and the proper unit tests 
> introduced). 
I agree there's a lack of consensus.  But I dont' believe that distutils 
is a strong basis for growth.  Distutils may be a reasonable choice of a 
build tool, (I'm not sure yet), but it's packaging and distribution 
support is minimal to nonexistent.

I think what's needed here isn't a consensus, (which we'll never get), 
but rather a generational solution based not on available technology, 
but rather on state of the art requirements.

For example, distutils says nothing about host packaging systems nor 
distribution.  The distutils doc doesn't really even broach the topic of 
impure python packages or distribution.  I don't know, for instance, 
whether C extensions are expected to be contained in a collection of 
different tar.gz archives or if there's expected to be one "fat" one 
which contains binary C extensions for a collection of different 
operating systems.

The new system needs to support mass updates of all installed packages, 
querying of installed packages, removal of installed packages, recursive 
instances of the packaging database, (perhaps with something like 
virtualenv), as well as all of the things that easy_install supports 
now, and most of the features that debian/rpm/etc support now including 
virtual packages, dependencies, suggestions, conflicts, and both command 
line and gui installers.  It should also be capable of at least "playing 
nice" with the host packaging system, (eg, rpm, deb, ipk, etc), or 
perhaps of completely excusing itself from the distribution process, 
(other than supporting the native distribution process).  If necessary, 
we can create extra repositories for those systems such that python 
packages can be distributed through the native packaging system, even 
though we're managing our own release processes.  I know that 
debian/ubuntu/rpm/easy_install all allow for this as it's very common 
for enterprises to use such a solution for distributing their own 
packages in-house.

The new system should also support "cross compilation" to the extent 
that such is needed for C extensions and to the extent that the native 
packaging system supports.  (Eg, debian doesn't, really but bitbake/OE 
virtually requires it).

Most of what I'm talking about here speaks to packaging formats, 
distribution processes, and installation processes.  And this isn't new 
technology.  Both debian, rpm, and several other unix technologies have 
fine systems in operation right now.  Sure, they all have weaknesses, 
but they are much better than easy_install.

--rich