[Distutils] Initial auto-installation support
Phillip J. Eby
pje at telecommunity.com
Mon May 30 23:54:54 CEST 2005
At 04:10 PM 5/30/2005 -0500, Ian Bicking wrote:
>I've added some very initial support for automatic installation of
>packages in Paste, using easy_install
>(http://peak.telecommunity.com/DevCenter/EasyInstall). In a
>configuration file you can put:
> use_package(package_name, pkg_download_url=None)
>And a couple other options, but we'll ignore those. It will look for
>the named package, and if not found will install it (generally in
FYI, just so you know, your implementation won't handle nested
dependencies, nor allow specifying optional features of requested packages.
I'd suggest that in a later version, you take a look at subclassing
pkg_resources.AvailableDistributions, and overriding the 'obtain()' method
to do the search and installation. In this way, your 'obtain()' method
will get called for any dependencies that the originally-requested package
To see what I mean, take a look at the source of pkg_resources.require(),
which looks like this:
requirements = parse_requirements(requirements)
for dist in AvailableDistributions().resolve(requirements):
So, if you subclass AvailableDistributions to define an 'obtain()' method,
and then write a similar loop using that subclass, you'll be able to
cleanly integrate the auto-download in a forward-compatible way for
packages that declare dependencies in their PackageName.egg-info.
>There's another aspect to Paste installation, where some packages
>(plugins) need to write things into Paste. I'm not sure quite how that
>will work -- maybe use_package() will see if there's a paste_install
>module in the package somewhere, and call that somehow.
That might be a good candidate for egg metadata; if a package is a Paste
plugin, you could require a 'paste-install' file in the package's
EGG-INFO. Note that the AvailableDistributions().resolve() method returns
a list of Distribution objects, and distribution objects have a 'metadata'
attribute that implements 'IMetadataProvider'. So,
'aDistribution.metdata.has_metadata("paste-install")' will tell you if that
distribution has a "paste-install" file in its EGG-INFO, and using the
get_metadata() method will give it to you as a string. Thus, you can do
for dist in MyDownloadDistributions().resolve(requirements):
If you use this algorithm, it will work for all egg varieties: compressed,
uncompressed, and "development" eggs. You will, however, need to keep
track of which paste-install scripts you've already processed, because
'resolve()' can yield distributions that are already present on
sys.path. Also, you may want to create and cache a single instance of your
MyDownloadDistributions class, because creating one does a lot of
filesystem stats and listdirs and such.
Of course, if you have people doing TheirPackage.egg-info/paste-install,
they can also create TheirPackage.egg-info/depends.txt, and list all their
dependencies there, with no need to use 'use_package'. It won't help with
download URLs, though. But we could perhaps define an
EGG-INFO/download_urls.txt as a stopgap, that lists known download urls...
Anyway, as you can see, eggs were definitely designed with plugin systems
like this in mind. :)
> But besides
>that, this should work now for any packages with a distutils install, so
>long as those packages are reasonably well behaved. Hrm... except
>setuptools 0.3a2 doesn't have SourceForge download support, but 0.3a3
>does and I think PJE will release that soon.
Hopefully within the next 24 hours or so. It will also include sandboxing
support (automatically aborts the install if the package tries to write to
the filesystem outside the build directory), and lots of workarounds to
support various packages out there that have quirky install_data subclasses.
Those items are already done, but items still on my to-do list include:
* help message to explain how to use require() for multi-version/instdir
* a --build-dir/-b option to set the build directory, that will leave the
downloaded package and its extracted contents in place after the
installation is complete. (So you can read docs, install scripts, or debug
a failed installation.)
And I'd like to do something about scripts, but I think that's going to get
left to an 0.4a1 release, assuming there are no further bug fix releases
needed in the 0.3 line.
More information about the Distutils-SIG