[Catalog-sig] The KISS principle [ Re: CPAN functionality for python - requirements ]

Robin Dunn robin@alldunn.com
28 Feb 2001 08:55:18 -0800

> But writing a "perfect" utility for automated download-and-install, with
> dependency tracking, etc etc, is going to be VERY HARD. Don't get misled -
> Perl doesn't have such a beast. And We won't even have what Perl has if we
> focus on perfection rather than practicality.

Agreed.  There is an old saying about shooting for the stars to reach the
moon.  Some of the ideas and proposals for all this seem to be shooting for
distant galaxies and are unable to even get off the ground.

IMHO we should start with something simple that allows for some of the
ultimate goals, and that can be built upon later to get to more of the

Here is what I think the simple version needs:

1. A standard place where a file containing a list of mirrors and maybe a
bit of mirror meta-data can be fetched from.

2. A python module to parse that mirror data into a meaningful data
structure such as a dictionary or a list of instance objects of some type.

3. At every site in the mirrors file there will be a file containing
meta-data about all the packages in the archive.

4. A python module to parse the package meta-data into something the client
code can easily use.  (The meta-data format is probably XML, but hiding it
like this means the tool developer doesn't need to know or care what format
the file is in.  Ditto for the mirror list.)

5. An automated way to extract at least most if not all of the package
meta-data from a Distutils based package and upload the meta-data, the
sdist, and any bdists that the developer has made to the archive.  This
could be a new command added to Distutils, with a bit of functionality on
the server side for receiving the files, moving them into the "right place"
in the archive, and updating the package meta-data file.

6. Define a file format and supporting Python code for tracking which
packages and versions have been fetched from the network and installed.
Again, this should probably be a function of Distutils so it will also catch
the cases where you downloaded some package not in the archive network and
ran "python setup.py install" yourself.

That's it.  With just that bit in place intelligent clients could be written
as command-line or GUI tools, or even a web-based interface.  They simply
fetch the list of mirrors and fetch the package meta-data from a desired
mirror and cache this info locally.  Then the client tools can do things
without having to talk to a server like list available packages, query
dependencies, list packages you have installed, etc.  When you want to fetch
and install a package then the client tool can use the package meta-data and
urllib to fetch the sdist and/or a desired bdist and then optionally install
it either using the package's setup.py or by whatever command is neccessary
for the bdist.

Robin Dunn
Software Craftsman
robin@AllDunn.com       Java give you jitters?
http://wxPython.org      Relax with wxPython!