[Python-Dev] setuptools in 2.5.

[Phillip J. Eby]

At 11:53 PM 4/20/2006 +0200, M.-A. Lemburg wrote:
>Only after endless discussions, Phillip added some weird
>--long-option-name-that-no-one-can-remember to the setuptools
>"install" command that restores the standard distutils behavior
>which should be the default anyway.

And later, I also made it the default behavior for installation with 
--root, without any further prompting from you or anyone, in fact.


>The suggestions that were made were constructive. Just look
>at the distutils-sig archive. Only very few of the suggestions
>were taken into account.

Actually, *all* of your suggestions were *taken into account*, by which I 
mean I have done all I can to understand them and to understand the 
concerns behind them -- as I do with all requests and feedback.  Some, 
however, will be a long time coming in implementation, or are things I 
simply can't agree with.  Those suggestions may or may not be "right" in 
some sense, but they are not right *for setuptools*.

But I take all suggestions into account, and sometimes I find ways to 
implement them.  When I say "no", it simply means, "I don't see how I can 
do that without breaking some other constraint."  It has happened 
repeatedly, however, that a way of doing something occurs to me months 
after the request, and then it is added with little fanfare.

Meanwhile, I'm always going to give a higher priority to the requests that 
are in strong alignment with the goals of setuptools, and come from:

* developers actually using setuptools,
* users who are installing packages that use setuptools, or
* system packagers who would like to package things that use setuptools

It also helps if they are things I can actually come up with a way to 
implement, without violating setuptools' goals.


>  Instead, setuptools fans insisted on
>their right to have everything "just work" in their sense
>of the word.

Yeah, but they're not as bad as those Python fans with their crazy 
insistence on significant whitespace.   ;)

Seriously, do you really think this is a *bad* thing?


>Most of them probably don't even know what the distutils
>or Python standards are for installing packages, where the
>motivation to have zip imports came from and have only
>ever seen and used the setuptools way.

Um, yeah, because Jim Fulton is such a newbie to Python, surely *he's* 
never had to use Makefile.pre.in to build exotic C exten...  Oh, wait, 
never mind.  :)


>The two camps need to come together again, but that will require
>understanding some of the history and standards that we've
>had in Python ever since Python packages were introduced.
>It will also require the setuptools folks to get some
>feeling of respect for those who have worked in the field
>for years.

I'm sorry, but this is an example of the highly patronizing attitude you've 
been giving on the distutils-sig since day one.  As it happens, I too 
remember the world before distutils.

With due respect, Marc, not every Python developer is you or Fredrik or 
Jim.  There are plenty of people with things to contribute to Python who 
find the distutils upsetting, confusing, or worse.  When I started doing 
work for OSAF, I was very surprised to find out how many developers new to 
Python found the distutils not merely inconvenient (which was my personal 
perspective) but simply unusable.  Some let forth streams of profanity at 
the mere *mention* of the distutils.

So, please consider this: to experts such as you and I, the distutils may 
merely be inconvenient or tedious, something to throw together an 
mxSetup.py or zpkgutil or some such convenient utilities to handle all your 
personal distribution needs and avoid the repetition and tedium.  And in 
fact, setuptools originally began as just such a few extensions for my 
personal use.

But to Python newbies, the distutils is a confusing, broken, undocumented, 
arcane, archaic, unpredictable useless piece of ****, an embarrassment to 
Python, far worse than Java, and many many other things.  And yes, those 
are all things that people have actually said to me.  They really, really 
want things to Just Work, and they say so in so many words.

So when I work on setuptools, I imagine those developers, and I try to 
imagine what they will think and what they will expect.

To be frank, you haven't shown in our discussions that you respect or value 
*those* developers' perspective.  You seem to believe that there are other 
things more important than making things Just Work for this audience.  I 
don't know what those things are, but all I can say is, to me they don't 
exist.  I just can't see it.  Because all I can see is the frustration on 
the faces of developers who were enthusiastic about Python until they 
encountered the distutils.

Distutils *is* great.  It's also unusable for developers who have better 
things to do with their time.  The two are not mutually exclusive, as 
Fredrik has often enough pointed out regarding other problems with Python's 
"public face".


>You may not know it, but having worked with
>distutils for a long time, I have great respect for
>Phillip's work - it's just that I find some of his design
>decisions wrong, since they don't follow the standards.

And that's where we disagree - standards exist to serve users, not the 
other way 'round.  Standards change when times change.  The choice to 
follow a standard is an economic decision for me, not a religious 
one.  Where distutils works for setuptools' goals, I embraced it, and where 
it does not, I extended it.  Call me Bill Gates if you must, but it is the 
*reason* that setuptools is successful, not blogosphere hype (you'll find 
almost as many anti-setuptools blog articles as pro- ones; the majority are 
actually just neutral) or screencasts and pundits (there are none).

It has "fans" because its design choices are strictly Darwinian: whatever 
makes setuptools be adopted by more packages is the right choice for 
setuptools.  Any other considerations have been secondary.  As Guido 
pointed out earlier, "acceptance is critical" for a system like 
this.  That's why the rallying cry is that things should "Just Work" -- 
because that's what people want.


>There's really nothing wrong with the standard distutils
>two step process:
>
>1. download and unzip the source file
>
>2. run "python setup.py install"

First, that's three steps.  "Download and unzip" doesn't come in one-step 
form so far as I know.

Second, you're leaving out the part where you upgrade a package that 
renames or moves or removes a module.  Your response on the distutils-sig 
was that the package author should write a distutils extension to do the 
necessary cleanup, which again highlights a complete lack of respect for 
package authors' time.

Not everyone who can contribute something valuable to Python's library base 
is willing to or even *can* become a distutils expert.  Simple use cases 
like upgrading or even uninstalling a library *should be simple*.  You 
should not have to be a packaging expert to share your package with the world.

I'm not saying setuptools is perfect for this either -- hence the 0.6 
version number for the stable release.  But it's the only thing that's even 
*trying*.


>I've been publishing the mx Tools since 1997. Back then we
>only had the old Makefile.pre.in mechanism and still most people
>managed to get the tools working (step 2 then being:
>"make -f Makefile.pre.in boot; make; make install") without
>having to ask for help. Since I've started using distutils
>in 2001, the number of support questions related to compiling
>and installing the tools dropped to near zero.
>
>I think this is quite a success story for distutils.
>
>Unfortunately, this fact is rarely mentioned in all these
>setuptools discussions, probably because it's just not
>the latest greatest and shiniest tool in the world
>anymore as it was perceived in the early days.

I wish people would stop interpreting setuptools as some kind of a dig at 
the distutils.  Does the existence of Python 2.5 mean that Python 2.4 is 
bad?  Does the existence of third-party libraries make the standard library 
incomplete?