[Distutils] A prefix option?

[Phillip J. Eby]

At 06:40 AM 2/10/2006 -0500, Jim Fulton wrote:
>Phillip J. Eby wrote:
>>At 07:31 AM 2/9/2006 -0500, Jim Fulton wrote:
>>
>>>I really don't think the virtual python approach is viable, at a minimum
>>>because it doesn't work on windows.  It is also unacceptably heavy IMO.
>>
>>What do you mean by "heavy"?
>
>It essentially duplicates the python install via symbolic links.

That's just a statement of what it does.  I was looking for an explanation 
of what makes that "heavy".  :)  One assumes that on Unix, directory 
entries and inodes are relatively cheap.


>>PYTHONPATH-based installs allow all of this, as long as you add the 
>>setuptools .egg to PYTHONPATH.
>
>Agreed. I think PYTHONPATH-based installations are close to what some of us
>need.  I think that having to set the path is a hassle that is possible
>to overcome and worth doing.  As I've suggested before, since easy
>install generates startup scripts, it should be possible to for it
>to generate startup scripts that take care of setting the path.
>
>I think it should be possible to have Python startup scripts invoke
>setuptools' site.py to add necessary .pth files to the path.

The regular site.py suffices for that.  What it doesn't suffice for is 
getting those directories on *ahead* of the system-provided libraries.  If 
you don't care about that, then adding a series of statements like this to 
the script would suffice:

     __import__('site').addsitedir("some target dir")

Where the series of directories represents your "instance", in the order of 
highest to lowest precedence.

Of course, this is unnecessary *except* for obtaining the location of 
setuptools itself, unless the goal is to have plugins installed using .pth 
files, and to manage that setup externally to the actual programs.  (By 
which I mean, the programs themselves have no notion of a "plugin 
directory", they simply expect everything to be already set up on sys.path 
when they run.)


>   If I'm
>wrong, then another option is for setuptools to generate sh scripts
>on 'nix platforms, much as it generates exe's on Windows.  (I presume
>that the generates exe files on windows then invoke Python and can
>also manipulate the environment.)

They could indeed manipulate the environment, though they'd need more info 
about it.  On 'nix platforms it's possible to actually embed a shell script 
in the beginning of a Python file with a bit of quoting cleverness.


>>>- is simple. :)
>>
>>PYTHONPATH is simple everywhere but Windows, and it's complex there only 
>>because you have to right click a bunch of icons to get to the editing 
>>facility.
>
>Eek.  I would never consider setting the PYTHONPATH that way on Windows
>to satisfy this use case.  The fact that you suggest this suggests that
>you missunderstand my use case.

Well, you haven't really explained your use case enough for me to 
understand even what these programs are or what people are doing with 
them.  You just listed some requirements.



>My goal is to have self contained working directories.  In addition,
>I want to have chained self-contained working directories, which is
>why the excellent "put scripts and eggs in the same place" solution
>doesn't work for me (more below).

You could generate a sitecustomize.py in a scripts+eggs directory, and have 
it do:

     import site
     site.addsitedir("this directory")
     site.addsitedir("another directory")

etc.  This would basically chain the directories in a way that supports 
.pth files.  Then just easy_install scripts and eggs to the single target 
directory, including the target directory in --site-dirs so easy_install 
knows it can use .pth files.

Presto - you're done.  Running a script would now suffice to set its 
environment.  Note that anything it references via .pth files will be 
*subordinate* to the same things in site-packages, but since you're 
presumably controlling what Python install is used for this, and in any 
case the scripts have require() used to bump up anything the script 
directly requires, this shouldn't be a big issue.

Note that in this setup, sitecustomize.py is the only configuration file 
you need, and you don't need any upgrades to setuptools as it exists today 
to make this work.  Just slap sitecustomize in the same directory with the 
scripts and eggs, and all is well.  You can, if you wish, modify it to do 
other sys.path munging according to your needs.

Unfortunately, due to differences in path processing on Windows and 'nix, 
the above procedure appears to only work correctly on Windows.  On 'nix, 
the current dir and/or script dir aren't on sys.path soon enough.  *sigh*.

Okay, I give up.  PYTHONPATH manipulation is the only thing that 
works.  After further experimenting, it seems the only consistent way to 
get sitecustomize.py to work, on all versions and platforms, is to have it 
on PYTHONPATH when Python starts execution.  In which case, you might as 
well use the site.py hack and control everything with .pth files.

Now you've got me wondering if maybe I should always have easy_install 
create script wrappers that set a fixed PYTHONPATH.  The main problem I see 
with it is what happens when somebody deliberately sets a different 
PYTHONPATH.  Do you append to it?  Prepend to it?  Ignore it? Should 
PYTHONPATH munging be optional?  Mandatory?  Hrm.

Ignoring PYTHONPATH when running a script has security advantages, to be 
certain.  Putting the fixed paths in front of the user-supplied ones is 
almost as good.

This needs a lot more thought, though.  Unfortunately, it's making me aware 
of some deficiencies in the existing script system, which doesn't allow for 
e.g. running scripts with -O at the moment, let alone any other 
options.  These are allowed in "traditional" distutils scripts, but not 
entry-point based scripts, which have no place to specify these things at 
the moment.  I'm probably going to have to add a way to control these and 
other script-generation options at some point.