[Distutils] Package install failures in 2.6.3 - setuptools vs Distribute

[Ned Deily]

In article 
<94bdd2610910030649r431a5638y7c8b5332934f5ae at mail.gmail.com>,
 Tarek Ziad? <ziade.tarek at gmail.com> wrote:
> On Sat, Oct 3, 2009 at 2:15 PM, Lennart Regebro <regebro at gmail.com> wrote:
> > 2009/10/3 Ned Deily <nad at acm.org>:
> >> I'm afraid there is going to be a small deluge of very confused users
> >> who will end up needing to install Distribute but only when they
> >> eventually figure out why some packages with C extensions mysteriously
> >> no longer install after they upgrade to python 2.6.3.  For example,
> >> following the package instructions to use setuptools easy_install, an
> >> easy_install lxml fails with 2.6.3 with the very cryptic:
> >>
> >> File "build/bdist.macosx-10.3-fat/egg/setuptools/command/build_ext.py",
> >> line 85, in get_ext_filename KeyError: 'etree'
> >>
> >> http://stackoverflow.com/questions/1512530/cant-install-lxml-python-2-6-3
> >> -osx-10-6-snow-leopard
> >>
> >> Appscript is another package that fails similarly.  There must be others.
> >
> > I can confirm this. It seems to be packages that use c-extensions.
> > zope.interface also breaks.
> 
> Yes Setuptools makes some assumptions on how distutils APIs are called
> and in which order. Once bitbucket is up, you can look at the details
> on the fix
> I did, it's fairly simple.
> 
> Notice that this has been fixed in Ubuntu already with a patched version
> of setuptools, but the next probable step is to use Distribute there
> and in debian.

That's fine but how does this help users on OS X, for example, when the 
python 2.6.3 they are using is supplied only by the python.org 
installers?  And the maintainers of the packages that can't be 
installed? 

> >>  $ curl -O http://nightly.ziade.org/distribute_setup.py
> >>  $ python distribute_setup.py
> >
> > Distribute doesn't break, though, so "sudo easy_install Distribute"
> > seems to be enough. :-)
> Yes that's the simplest way to fix the problem.

But that's assuming they have a working easy_install installed for 
*that* version (2.6.3) in the first place.   For instance, the OS X 
installer does not include setuptools or Distribute.  I'm concerned 
about users who just want to install some third-party package for the 
first time.  They try to follow the recipe for bootstrapping Distribute 
and fail right at the first step because they don't have wget on their 
system (for example, OS X) and they don't know what it is.  Getting it 
right for beginners is one of the reasons why easy_install was developed 
in the first place.

> Notice that setuptools patches distutils in other places, and it will
> not work with Python 2.7 and 3.2
> where Distribute does.

That's fine, too, and there are lessons to be learned here, I think, 
that should influence 2.7 and 3.2.  But the big problem is distutils in 
2.6.3. This is not trying to debate the Distribute fork; the target 
audience for easy_install doesn't care one way or the other. More 
importantly, they haven't a clue that there is something called 
Distribute and they don't know what setuptools is.  Like it or not, 
easy_install / setuptools is a major customer of distutils and somehow 
it got broken for 2.6.3, a point release.  From the end user and the 
3rd-party packagers' points of view: the 3rd-party package didn't 
change, setuptools didn't change (obviously!), but now the package won't 
install and, worse, fails cryptically.  So the problem must be Python - 
so they think and with some justification, it seems.  Once they get 
Distribute installed, everything should be fine but how do they know 
they are supposed to install Distribute?

This is not a good experience for users.  Unless I'm missing something 
(and I hope I am), this issue really can't be hand-waved away.

-- 
 Ned Deily,
 nad at acm.org