[SciPy-dev] issues with installing Numeric in a user directory

[eric jones]

I have been building/installing 3rd party python (numeric/scipy/etc.) as
a user instead of as root lately.  This means the packages are installed
in a non-standard place (like /home/eric/linux or /home/eric/sunos).  I
do this using:

    python setup.py install --prefix=$OSDIR

where OSDIR is an environment variable set to specify a platform
dependent path (my home directory is shared between about 6 different
architectures).

The user (instead of root) installed locations have caused several
annoyances, one with include directories and one with .pth files.  They
have both manifested themselves when dealing with Numeric.  

(1) include directories.
Distutils knows to include files from /usr/include/python2.2 (or
wherever it is installed) whenever building extension modules.  Numeric
installs its header files inside this directory when installed as root.
However, when I install Numeric in /home/eric/linux, the header files
are installed in /home/eric/linux/python2.2.  Distutils doesn't know to
look in hear for headers.  To solve this, I'd have to hack all the
setup.py files for modules that rely on Numeric to use my include_dirs.
This isn't so nice.

To rectify the problem, scipy_distutils now searches for an environment
variable called PYTHONINCLUDE that uses the same syntax as PYTHONPATH
and PATH.  If PYTHONINCLUDE is found, the directories listed in it are
added to the include path during compilation of C/C++ files by
build_ext.py and build_clib.py.  On my machine, I specify 

export PYTHONINCLUDE=$OSDIR/include/python2.2

and all setup.py files work unaltered.  I think this is a reasonable
solution.  Has anyone solved this problem with an alternative method
that might be better.

(2) .pth files
Numeric is a "package" that doesn't really behave as a package.  It
dumps all of its files in site-packages/Numeric, but it doesn't define a
__init__.py file.  Instead, it also puts a Numeric.pth file in
site-packages that tells python to also search site-packages/Numeric for
modules.  This works fine when Numeric is installed as root, but it
doesn't work when it is installed in non-standard places.  This is
because .pth files are only expaneded when they exist in the "system"
locations.  They are ignored anywhere else in the search path.  As a
result, after installing Numeric with "python setup.py install
--prefix=$OSDIR", importing Numeric fails even if 

PYTHONPATH=$OSDIR/lib/python2.2/site-packages

The fix is easy for someone who understands what is going on -- add
$OSDIR/lib/python2.2/site-packages/Numeric to the PYTHONPATH.  However,
the uninitiated may flail around a long time trying to figure out why
installing a package to there standard location doesn't work --
especially if it has worked for every other package they have installed.

I haven't figured out a nice work around here, other than actually
making Numeric a real package which I doubt is even close to an option.
Does anyone else have a better solution?

Thanks,
eric

----------------------------------------------
eric jones                    515 Congress Ave
www.enthought.com             Suite 1614
512 536-1057                  Austin, Tx 78701