[Distutils] Non-root install for testing

[Robert Kern]

Ian Bicking wrote:
> I've created a script to set up a non-root installation of Python,
> suitable for testing installation procedures.  It follows the
> instructions located here:
> http://peak.telecommunity.com/DevCenter/EasyInstall#non-root-installation

I really dislike this approach. I'm afraid that I don't understand what
problem it's trying to solve. On OS X, the instructions tell you to
install to ~/Library/Python/2.4/site-packages . Why do the general UNIX
instructions tell you to make a ton of symlinks? IMHO, better advice
would be (thanks in part to Prabhu Ramachandran):

  Make a directory to hold your EasyInstall'ed packages (e.g.
  ~/lib/python2.4/site-packages). easy_install uses .pth tricks.
  Therefore, the directory where eggs are installed require that .pth
  files work there.  On non-system wide installs, this can be achieved
  using a sitecustomize.py somewhere inside PYTHONPATH with the
  following lines:

    import site, os
    site.addsitedir(os.path.expanduser('~/lib/python2.4/site-packages'))

  easy_install needs to be told to use this directory and that it is
  okay to use. Add the following options to your ~/.pydistutils.cfg
  file:

    [install]
    install_lib=~/lib/python$py_version_short/site-packages
    install_scripts=~/bin

    [easy_install]
    site_dirs=~/lib/python$py_version_short/site-packages

At this point in time, I think this approach is cleaner. Eventually it
might run into conflicts with a distribution-provided easy_install, but
AFAICT there aren't any of those yet.

In any case, neither of these approaches is going to be tenable in the
long term. Having the non-root user recreate the whole python
distribution via symlinks isn't a reasonable approach. Controlling the
installation/activation of eggs via .pth files was a cool approach, but
it appears that it's not terribly flexible because of the way that
Python inserts them into sys.path.

A more general way forward might be to have sitecustomize.py (or site.py
if your distro is kind enough to take care of such things for you) read
data files that list the eggs that are activated. sitecustomize.py could
then insert the eggs wherever it sees fit into sys.path. On Debian, for
example, there would be a file in /usr/lib/pythonX.Y/site-packages/ for
eggs that are provided by real dpkg-installed .deb's. There would be
another file in /usr/local/lib/pythonX.Y/site-packages/ for eggs that
are installed by root but aren't real Debian packages. There might be a
final one somewhere canonical in a user's home directory
(~/.easy_install perhaps) for eggs that are local to the user (and
probably installed to ~/lib/pythonX.Y/site-packages/ but not
necessarily). These data files would be searched in reverse order: ~/,
/usr/local/, /usr/. It would be nice if the data files could specify
that certain eggs in the later directories are to be ignored.

Does this sound like a workable idea?

On a similar note: easy_install does a lot of sanity checks to determine
if it ought to install to the given install_lib directory.
Unfortunately, those sanity checks can be too restrictive. For example,
when I use Debian, I manage my /usr/local/ tree with GNU Stow. I want to
put everything managed by easy_install under the prefix
/usr/local/stow/easy_install/ (with subdirectories bin/,
lib/python2.4/site-packages/, and so on). When I run Stow, everything
will be symlinked into /usr/local/. Because Debian was thoughtful,
/usr/local/lib/python2.4/site-packages/ is already on the PYTHONPATH and
is .pth-enabled so everything would work just fine. Unfortunately,
/usr/local/stow/easy_install/lib/python2.4/site-packages/ isn't on the
PYTHONPATH and easy_install will refuse to install there even when it
has been specified as a valid --site-dir.

Paranoia is healthy, but we need a way to override the paranoia. I am
requesting a --force option to override this sanity check. Preferably
there would be one --force option per sanity check and one overall
--force that entails all of the others, but I'll take what I can get.
I'm willing to work on a patch if given a little guidance as to where
all of the sanity checks are.

Thank you for your attention.

-- 
Robert Kern
rkern at ucsd.edu

"In the fields of hell where the grass grows high
 Are the graves of dreams allowed to die."
  -- Richard Harter