[Distutils] Distributing 'external packages'

[Martijn Faassen]

Hi there,

Recently in the discussion on autoconf I've seen it mentioned that lots
of Python extensions build on external non-python packages (that may be
configured by autoconf or in many other ways).

A developer using these external packages will obviously have them
installed, and it's not too strange to expect co-developers to download
these packages and install them as well.

However, now we get to the two other audiences of distutils: the
distributor/packager, who prepackages everything for one or more
particular platforms, and most importantly the user who doesn't want to
know about anything, just wants to run the program.

If I'm a plain user and want to try out Python-Powered XML-database
Warpdrive Enhancer version 4.2, I don't want to be required to download
and install the Warpdrive package, and the XML package, etc, if I can
avoid it. I just want to install and go.

For many systems of course the separate install is unavoidable; we can't
package Oracle for download, for instance. But for many smaller packages
(such as libraries) that may or may not be on the user's system it
becomes more important.

So, as a reminder, how are we going to handle this? Some random
requirements and ramblings:

* The packager/distributor would like a standard way to somehow include
these packages (or at least check for them). This doesn't mean
*building* these packages, but it does mean a standard way to pack them
up and install them.

* We don't have any control over these external packages. Still, we
don't want everybody to grow their own way to deal with particular
packages.

* If two Python packages are installed that both use FooLib, we don't
want the disutils to install FooLib twice. Same if FooLib is already
there. That's also why we need a standard way to handle these.

* An idea I aired previously was to provide somekind of disutils wrapper
for common packages. For instance, if FooLib is often used by software
written in Python, we make a wrapper for it. The wrapper calls Foolib's
installation/configuration methods (which may for instance be rpm
commands, or a windows installer program) where necessary:

   * Install FooLib
   * If this is not possible without manual intervention, some way to
tell the
     user what to do in easy steps.
   * Check if FooLib is already there (and what properties it has)
   * Also check if is it already registered with a Disutils Wrapper; if
not, 
     some way to try to add the wrapper so that the next pyapps that get
installed
     won't need to go through this process again.
   * Possibly also uninstall FooLib, though this may be far too tricky.

Eventually if this gets popular a Disutils Wrapper can even be
distributed with FooLib itself, but the Disutils Archive should also
offer some consistent way to get at known external package wrappers.

All of this may be simply too hard, as packages can vary in many ways,
but I think there are least *some* common things one can standardize.
The idea is to hide all the nonstandard ways behind some standard
interface, as far as possible, to help packagers and developers.

It occurs to me that all of this is somewhat analogous to autoconf
again; autoconf checks out capabilities of the system, in particular of
the C compiler, and libraries. But as far as I know autoconf isn't
modular as this package approach would be.

Now you all start shooting this down. :)

Regards,

Martijn