[Distutils] Package installation strategy !

[Greg Ward]

On 23 October 2000, Michel Sanner said:
> Hello,
> 
> I had a look at Python2.0 over the week end and I noticed that the
> installation procedure creates a site-packages directory under $prefix
> but not under $exec_prefix.
> 
> Is there a consensus about wher packages containing a mix of .py and
> .so files should go ?

There's not just a consensus on this: there is an iron-clad, hard-coded,
disobey-it-at-your-peril policy dictated by the BDFL.  Specifically,
"impure" module distributions are completely installed under
exec_prefix.  I believe this is documented in the new "Installing Python
Modules" manual.  (If it's not, that's my fault.)

> Should we ise the PyOpenGL strategy of extending the path with a
> sub-directory for platform specific stuff at run time ?

Absolutely not.  I think it's pretty much agreed that this is madness.

> or should we
> place all .so files in lib-dynload ?

Absolutely not.  That directory is for standard extensions, not
third-party additions.

> or should we have a site-packages
> under $exec_prefix and duplicate the .py files (my concern here is not
> disk space but rather keeping these files in sync!)

This is the same problem as with C libraries -- if there are N copies of 
libfoo.a installed (for N architectures), should there be N copies of
foo.h, or 1 copy?  The traditional answer is to save the great expense
of (N-1)*10k (or however big foo.h is) and keep 1 copy.  Obviously disk
space is no argument nowadays, but the synchronization thing still comes 
up.

This argument is a red herring.  In fact, life is *harder* if you keep 1
copy of foo.h (or foo.py) for N copies of libfoo.a (or foo.so), and
here's why: imagine that foo 1.0.4 is installed for Linux and Solaris.
Everything works fine for many months.  Then someone comes along and
installs foo 1.1.2, but only on Linux.  Now your Linux directories have
the 1.1.2 version of libfoo.a and foo.h, and your Solaris directories
have libfoo.a from 1.0.4, but foo.h from 1.1.2.  You *lose*!  If you
kept separate copies of the header file for each architecture, then
you'd still have the minor headache of different versions of foo, but at
least foo would be internally consistent on each architecture.  When you
try to share files, though, then *you* *will* *lose*.

Based on a true story... (hello, Jeremy!)

> I know I have already brought this up ! so forgive me .... but I
> rather have a mechanism allowing Python to search platform dependent
> extensions first (to get fast versions if they are available) and then
> search platform independent packages even if a package match has be
> found earlier.

If you want foo.so under exec-prefix and foo.py under prefix, then the
will have to be separately distributed.  If there are *any* extensions
in your module distribution, the entire thing is installed under
exec-prefix.  I think in this case foo.so would shadow foo.py, and
anyways this exec-prefix had better not be visible to hosts that can't
load foo.so!

        Greg
-- 
Greg Ward - maladjusted nerd                            gward@python.net
http://starship.python.net/~gward/
If you can read this, thank a programmer.