[Distutils] EasyInstall: bootstrapping
Phillip J. Eby
pje at telecommunity.com
Sun Jun 5 00:08:51 CEST 2005
At 04:05 PM 6/4/2005 -0500, Ian Bicking wrote:
>Phillip J. Eby wrote:
>>Hm. I would suggest that the safest tack to take is try importing
>>pkg_resources and do a 'require()' for the setuptools version that you
>>need. If the import fails or the version fails, bail out of setup.py,
>>asking for the user to install the correct setuptools. I think that
>>trying to invoke easy_install recursively or guess where the egg should
>>be installed is a bad idea. (E.g., guessing about the egg location will
>>cause easy_install's sandboxing to shut you down anyway)
>>However, it would certainly be nice to have a way for setuptools-using
>>packages to bootstrap the setuptools install -- and that includes
>>setuptools itself. (My head is starting to spin now.)
>>Hm. Unfortunately, it seems that the only other sane way to do this is
>>to distribute Paste as an egg, and require people to use easy_install to
>>install it, because then any dependency back to setuptools can be
>>resolved properly once easy_install's automatic dependency installation
>>is in place.
>I wouldn't mind so much requiring easy_install to be installed -- if
>that's the only firm prerequesite, that's okay. But a Paste checkout
>gives you docs and examples and unit tests, which aren't part of an egg,
>so Paste as installed by easy_install isn't complete.
Well, there's always -b, as of 0.3a3, which keeps the unpacked source
distribution after installation. Does that help?
>One option would be if there was an option to install something the same
>way you recommend using a development checkout -- don't create an egg,
>just make sure the PackageName.egg-info directory is in place.
Ugh. I don't want to do that for stuff that isn't either a development
checkout or part of the standard library. Anywhere in between and you
start to mess up version management. Note that when an .egg-info directory
is in a directory on sys.path, it means the egg in question is *already* on
sys.path, which means you can't require() a version that conflicts with it.
So, .egg-info is really a trick for development. EasyInstall already takes
care of everything you need for installing as an egg directory, so
installing .egg-info doesn't really provide any benefit that I see.
Now, if somebody wants to manually set a .pth file (or PYTHONPATH et al) to
point to a Paste checkout and its included .egg-info, that's fine: just add
the .egg-info directory to subversion after you've generated it, and you're
all set. This is in fact recommended for development scenarios. You just
don't want to try and do this by actually *installing* the package and
.egg-info to site-packages, because then you can't get it back out again.
> Incidentally, should there be a egg_setup command or something short of
> creating an egg you won't use to create that special directory?)
I guess I could factor it out into an 'egg_info' command, and call it from
'bdist_egg'. It might take some work to prevent duplication, though, and a
big problem is that it wants to build a 'native_libs.txt' in the info
directory, which it has to get the information for from the install_lib
command. But I could maybe not generate that file in the 'egg_info'
command, but then there's duplication of process in bdist_egg to find the
No, it doesn't make any sense to do all that extra work. The idea that the
egg is something "you won't use" is an illusion in the long run, and
meanwhile the build process helps validates the correctness of your
``setup.py`` and much of the rest of your package. The disk space and CPU
to build the egg should be minimal, and you only need to do it again if you
change your package's metadata (e.g. its version). The rest of the time
you can use it as-is from subversion or CVS. So just use bdist_egg.
>Also, if I was just going to get rid of paster install (which is fine, no
>big investment there), it does have the plugin-enabling code (where
>plugins can register themselves with Paste). I'm not sure where that
>would go; you could put the code into plugins themselves -- very
>reasonable -- except how do they find the appropriate Paste
>installation? I hate PYTHONPATH, and that's where this would lead.
I don't understand what you're trying to do here. Why do plugins need to
find Paste? Shouldn't Paste find the plugins?
My general approach to application plugin installation is that applications
have a plugin directory where the plugins' eggs get dropped (either via
EasyInstall, or by the app itself). The application then just uses
``find_distributions(directory)`` to iterate over the distributions and
check for plugin metadata, and do any registering of services provided by
the plugins. (Or, you can use the ``AvailableDistributions`` class to
automatically filter, index, and sort the eggs found.)
For larger application frameworks comparable to Eclipse in the Java world,
those apps would likely keep a registry of already-installed plugins and
their registered services, and thus would only scan the plugin directory to
locate newly- installed plugins. Some applications might also have an
explicit installation process of their own.
Anyway, as you can see, there are many architectures you can use to manage
application plugins using Python Eggs. I should probably expand on the
above a bit and add it to the docs, but I'd really like the rest of
EasyInstall to be done, so that it can use EasyInstall as an example of
downloading packages and resolving dependencies automatically. (Although
in truth most application plugin systems will probably want to just
download eggs directly rather than building them, and have some
application-centric directory of download URLs.)
>I almost think it makes sense to just have a hard line for packages that
>use this stuff: if setuptools isn't installed (or is too old), then go
>install it and start again. That way, assuming it is popular, it can
>become kind of standard regardless of your Python version.
That's my inclination as well. I suspect that web application deployment
is a killer use case for eggs, too. Certainly it was the use case (besides
Chandler) that pushed me to take action on designing and implementing
More information about the Distutils-SIG