[Distutils] packages and Maintaining Python in general

Greg Ward gward@cnri.reston.va.us
Tue, 19 Jan 1999 08:57:44 -0500

Quoth Sanner Michel, on 18 January 1999:
> I have spend my week end moving opn from Python1.5 to Python1.5.2b1 and
> have
> come to the conclusion that there OUGHT to be better way !!!

Yeah, I came to that conclusion within about two weeks of starting to
use Python (last September).  All it took was trying to find the correct
version of NumPy (the Matrix SIG web page at that time pointed to Jim
Hugunin's old ftp space at MIT, which is a tad out-of-date), and then
build it.  It wasn't impossible, it was just clunkier than it ought to
be.  I whined about this in the presence of Fred Drake and Andrew
Kuchling, they both agreed, and ultimately this SIG was created.

> 1 - I started to use Makefile.pre.in to generate make file for
> additional packages. Currently when any user types "python" he gets
> 1.5 (current public version). So I decided to build Python1.5.2b1 and
> install it somewhere not in the users path. The problem is that when I
> use Makefile.pre.in it uses "python" and gets version 1.5. The only
> work around I found was to edit the copy of Makefile.pre.in and make
> it point to the 1.5.2b1 binary.

This will definitely be addressed by the Distutils suite:
Makefile.pre.in will be a thing of the past, replaced by a (usually)
tiny Python script.  Normally this will just be run as "./setup.py", but
of course you can always say "python1.5.2b1 setup.py" and the build
process will follow the defaults associated with *that* particular
binary (eg. compiler flags, installation directories, etc.).

> 2 - When building Python for several architectures, the installation
> script allows to separate platform-dependant and platform-independant
> files.  I'd like the same for packages, i.e. when a package has both
> types of files I'd like to be able to have a package with that name in
> the platform-independant AND in the platform-dependant tree. This DOES
> NOT work currently because import stops searching after it find a
> package matching that name.

I completely agree.  I think this will require some cooperation on the
part of Python's configure/build process, and probably the startup code
that initializes sys.path.  So it's only likely to happen when/if a)
Distutils comes to pass as planned, b) Python 1.6 goes out.  But it's
definitely an important thing to keep in mind.

> 3 - Numeric Python should really be part of the standard distribution
> .. (please :). Actually I would even argue that it should be part of
> the Kernel such that developers can rely on the fact that this data
> type, CRUCIAL for any scientific application and key to Python's
> success in that field is available.  This would also allow for
> instance to have this datatype included in the Corba mappings for
> Python.

I disagree.  The whole point of the Distutils SIG is to make it trivial
to add third-party modules to an existing Python installation; if NumPy
being crucial for scientists justifies adding it to the core, then why
not add PIL for image-processors, and so forth?  Where do you draw the

> 4 - How difficult would it be to provide some more flexibility in the
> directory
> sctructure created by "make install":
> I'd like to have something like:
> PYTHONROOT = /mgl/tools/python/$VERSION
> $PYTHONROOT/include
> $PYTHONROOT/packages               # arch independent packages
> $PYTHONROOT/$ARCH_OS/packages      # arch dependent packages
> I do not need the Python version "python1.5" below that. In that way I
> can
> install any new version in parallele with an existing one.

Interesting proposal.  I think Python should support something like its
existing scheme (where its installation is scattered across lib, bin,
man, and include directories under $prefix and $exec_prefix) and
something like Michel's proposal (where it's concentrated under a single 
"Python root").  Of course, you can always hack the latter by saying
"./configure --prefix=/usr/local/python", but then you end up with the
library in "/usr/local/python/lib/python1.5", which is a bit weird.  You 
do get version discrimination on the binary, library, and include files, 
but at a pretty low level; there's a lot to be said for discriminating
early on, as in "/usr/local/python/1.5.2b1/...".  

Many issues to be discussed there.  Obviously have my opinions... anyone

Greg Ward - software developer                    gward@cnri.reston.va.us
Corporation for National Research Initiatives    
1895 Preston White Drive                      voice: +1-703-620-8990 x287
Reston, Virginia, USA  20191-5434               fax: +1-703-620-0913