[Distutils] Easy install, making "Traditional" PYTHONPATH-based Installation work better

[Phillip J. Eby]

At 10:52 AM 2/5/2006 -0500, Jim Fulton wrote:
>I'be been learning about setuptools and eggs and am very pleased.
>There is one area where I'd like to see a small tweak.
>
>In:
>
> 
>http://peak.telecommunity.com/DevCenter/EasyInstall#custom-installation-locations
>
>four options are presented for installing outside of Python installation.
>
>Naturally, I prefer the the last option, which the documentation recommends
>against. :)  Here's why. An important use case to me is the ability for
>a developer to work in a development sandbox without affecting anything
>outside the sandbox.  The solution needs to be cross platform.  The
>"Administrator Installation" option doesn't work for me because it requires
>modifying the Python setup.  Neither the "Mac OS X "User" Installation"
>nor the 'Creating a "Virtual" Python' options work for me because they
>are not cross platform.  I also find the "Virtual Python" approach a bit
>heavy handed.
>
>In playing a bit, I find the "Traditional" PYTHONPATH-based Installation
>works pretty well except for the problem of having to set the path manually
>when invoking scripts.  This problem could be addressed by having
>the generated scripts include the custom path settings.

Unfortunately, that's probably harder than you think.  The reason I advise 
against the PYTHONPATH-based approach is that it uses a sneaky hack to get 
site directory processing to work "properly" (i.e., as desired).

When you use the PYTHONPATH-based approach, it only works if you put the 
setuptools egg directly on the PYTHONPATH at *Python startup*, because 
setuptools includes its own version of the 'site' module that bypasses the 
standard Python 'site' module, so that it can scan .pth files that are 
*already on* PYTHONPATH.

This means that no Python code embedded in the scripts is going to affect 
this.  If you want to use PYTHONPATH, you have to set PYTHONPATH in the 
environment.  By the time the Python script itself is running, sys.path has 
already been manipulated such that it's no longer possible to process .pth 
files in a way that gives them precedence over system-installed packages.

Now, if you don't care about the .pth files and just want to use whatever 
the script's require() picks up, then it's not a problem.  However, since 
your post talked about .pth and site-dirs, I assumed that means you want to 
process that stuff.


>For now, I can create a wrapper around easy_install that does
>custom script generation, which could serve as a prototype for the
>proposed change.

Try it out, and see if you can get a privately installed package to 
*override* a system-installed one, without setting PYTHONPATH in the 
environment prior to Python startup.  I'll be surprised and intrigued if 
you can find a way.  (*Adding* packages this way is of course easy; it's 
overriding system-installed packages that's the problem, because you have 
to get the .pth stuff in *front* of site-packages in sys.path, and that 
normally doesn't happen.  It only happens with setuptools because of its 
hacked 'site.py'.)

If you don't care about .pth files, then you can simply install the desired 
eggs or egg-links to the same directory as the scripts, and PYTHONPATH only 
needs to include the setuptools egg.  It's either that, or perhaps put some 
bootstrap code in every script to locate setuptools if it's not already on 
sys.path.