[Distutils] Newbie questions about setuptools
Phillip J. Eby
pje at telecommunity.com
Thu Jun 21 15:09:55 CEST 2007
At 07:33 AM 6/21/2007 -0500, Edward Ream wrote:
>>It appears to me as though you do not have a project that can be
>>meaningfully installed or distributed using the distutils or setuptools,
>Doesn't this say more about distutils and setuptools than about Leo?
Depends on what you mean by "say". :)
Per http://www.python.org/doc/2.1/dist/ , the distutils were created
"to make Python modules and extensions easily available to a wider
audience with very little overhead for build/release/install mechanics."
They aren't for installing arbitrary applications. It just happens
that, if your "application" consists solely of scripts that tag along
with some libraries and immutable data, then they also work.
It's not reasonable to expect the distutils to be a general purpose
software installation toolkit; they are, after all, for sharing
>> > To put it another way, disutils and setuptools are designed to
>> install > *immutable* collections of code and data.
>The distinction between mutable and immutable data makes some (but
>not complete) sense on Linux. It is irrelevant on Windows.
Actually, Microsoft has been trying to get people to stop putting
mutable data in the \Program Files tree for many years now. That's
why user-specific $APPDATA directories exist there.
>Here is what I particularly like about setuptools:
>setup.py sdist register upload
>easy_install leo cleverly uses a generic upload_url
>setuptools.set can (at least sometimes) find all files managed by cvs.
>Why do any of the above require any particular organization of files?
They don't -- but a LOT of other features of easy_install *do* depend
on them. Notably, easy_install is intended to support (relatively)
clean uninstallation, and the simultaneous installation of multiple
versions of a project along the same PYTHONPATH.
Aside from locating files in the first place, most of easy_install's
complexity comes from either 1) supporting near-arbitrary
distutils-based projects or 2) managing sys.path. Your project
doesn't use either of these features, but they're absolutely vital
for setuptools' *real* goal, which is to make widespread library
sharing and reuse practical.
The distutils do fine for distributing individual projects, but does
badly for inter-project dependencies. This makes people who want to
depend on other libraries either bundle them inside their project, or
write their own replacement just to avoid the hassle of updating
their bundled versions of things. Then, you either end up with
multiple copies of some library installed anyway, or version skew/stomp.
Setuptools was intended to fix this by making it easy to say, "my
library uses these other libraries" -- and have them automatically
installed, with version skew problems handled. This is a delicate
enough problem to solve without throwing mutable files into the mix.
>We in the Leo community never argue over user preferences because my
>guiding design principle is this: if there are two or more even
>halfway reasonable ways of doing something, Leo provides an option
>that allows users to choose what works for them, and that's is the
>end of the discusssion. It seems that allowing me to distribute Leo
>as I and Leo's users prefer would be a useful addition to setuptools.
Projects that don't consist of packages, immutable data, and scripts,
don't really support library reuse very well. In fact, projects that
include mutable files, would be in direct *opposition* to
easy_install's goals for package management, so it really shouldn't
be surprising that it doesn't support them very well. :)
If you'd like to use the distutils or setuptools to distribute leo, I
would suggest restructuring it as a set of Python packages rather
than as a collection of modules and directories. (E.g., "import
leo.config" rather than "import leoConfig".) For plugin support, I
would suggest you investigate setuptools "entry points" system, which
many packages and applications use to automatically discover
This method has the advantage that people can distribute plugins as
setuptools-based projects, and users can install them with
"easy_install pluginprojectname". (See, for example, all the various
TurboGears widget plugin projects on the Cheeseshop, the TG/Buffet
templating engines, Chandler plugins, etc., for examples of plugin
systems built this way.)
I'm not saying this would be easy or simple; many existing projects
with other plugin or extension systems or using other distribution
mechanisms have taken some time to migrate or are still migrating to
this approach, in order to take advantage of the benefits. And
truthfully, most projects using the full spectrum of setuptools'
plugin abilities are relatively new, written after the features were available.
Certainly, you could also retain older plugin mechanisms, and scan
configured directories for plugins in addition to any "built-in"
plugins that you provide in your main distribution project, so that
you keep continuity for your existing users.
Of course, you may also decide that the cost/benefit isn't there for
you, and that's understandable. In that case, you may wish to use
some other distribution mechanism, or create your own; setuptools
unfortunately can't both achieve its own goals, and support
installing non-importable things (aside from scripts and immutable
package data) or arbitrary installation locations for project contents.
More information about the Distutils-SIG