[Distutils] A prefix option?
Phillip J. Eby
pje at telecommunity.com
Fri Feb 10 18:36:51 CET 2006
At 06:40 AM 2/10/2006 -0500, Jim Fulton wrote:
>Phillip J. Eby wrote:
>>At 07:31 AM 2/9/2006 -0500, Jim Fulton wrote:
>>>I really don't think the virtual python approach is viable, at a minimum
>>>because it doesn't work on windows. It is also unacceptably heavy IMO.
>>What do you mean by "heavy"?
>It essentially duplicates the python install via symbolic links.
That's just a statement of what it does. I was looking for an explanation
of what makes that "heavy". :) One assumes that on Unix, directory
entries and inodes are relatively cheap.
>>PYTHONPATH-based installs allow all of this, as long as you add the
>>setuptools .egg to PYTHONPATH.
>Agreed. I think PYTHONPATH-based installations are close to what some of us
>need. I think that having to set the path is a hassle that is possible
>to overcome and worth doing. As I've suggested before, since easy
>install generates startup scripts, it should be possible to for it
>to generate startup scripts that take care of setting the path.
>I think it should be possible to have Python startup scripts invoke
>setuptools' site.py to add necessary .pth files to the path.
The regular site.py suffices for that. What it doesn't suffice for is
getting those directories on *ahead* of the system-provided libraries. If
you don't care about that, then adding a series of statements like this to
the script would suffice:
__import__('site').addsitedir("some target dir")
Where the series of directories represents your "instance", in the order of
highest to lowest precedence.
Of course, this is unnecessary *except* for obtaining the location of
setuptools itself, unless the goal is to have plugins installed using .pth
files, and to manage that setup externally to the actual programs. (By
which I mean, the programs themselves have no notion of a "plugin
directory", they simply expect everything to be already set up on sys.path
when they run.)
> If I'm
>wrong, then another option is for setuptools to generate sh scripts
>on 'nix platforms, much as it generates exe's on Windows. (I presume
>that the generates exe files on windows then invoke Python and can
>also manipulate the environment.)
They could indeed manipulate the environment, though they'd need more info
about it. On 'nix platforms it's possible to actually embed a shell script
in the beginning of a Python file with a bit of quoting cleverness.
>>>- is simple. :)
>>PYTHONPATH is simple everywhere but Windows, and it's complex there only
>>because you have to right click a bunch of icons to get to the editing
>Eek. I would never consider setting the PYTHONPATH that way on Windows
>to satisfy this use case. The fact that you suggest this suggests that
>you missunderstand my use case.
Well, you haven't really explained your use case enough for me to
understand even what these programs are or what people are doing with
them. You just listed some requirements.
>My goal is to have self contained working directories. In addition,
>I want to have chained self-contained working directories, which is
>why the excellent "put scripts and eggs in the same place" solution
>doesn't work for me (more below).
You could generate a sitecustomize.py in a scripts+eggs directory, and have
etc. This would basically chain the directories in a way that supports
.pth files. Then just easy_install scripts and eggs to the single target
directory, including the target directory in --site-dirs so easy_install
knows it can use .pth files.
Presto - you're done. Running a script would now suffice to set its
environment. Note that anything it references via .pth files will be
*subordinate* to the same things in site-packages, but since you're
presumably controlling what Python install is used for this, and in any
case the scripts have require() used to bump up anything the script
directly requires, this shouldn't be a big issue.
Note that in this setup, sitecustomize.py is the only configuration file
you need, and you don't need any upgrades to setuptools as it exists today
to make this work. Just slap sitecustomize in the same directory with the
scripts and eggs, and all is well. You can, if you wish, modify it to do
other sys.path munging according to your needs.
Unfortunately, due to differences in path processing on Windows and 'nix,
the above procedure appears to only work correctly on Windows. On 'nix,
the current dir and/or script dir aren't on sys.path soon enough. *sigh*.
Okay, I give up. PYTHONPATH manipulation is the only thing that
works. After further experimenting, it seems the only consistent way to
get sitecustomize.py to work, on all versions and platforms, is to have it
on PYTHONPATH when Python starts execution. In which case, you might as
well use the site.py hack and control everything with .pth files.
Now you've got me wondering if maybe I should always have easy_install
create script wrappers that set a fixed PYTHONPATH. The main problem I see
with it is what happens when somebody deliberately sets a different
PYTHONPATH. Do you append to it? Prepend to it? Ignore it? Should
PYTHONPATH munging be optional? Mandatory? Hrm.
Ignoring PYTHONPATH when running a script has security advantages, to be
certain. Putting the fixed paths in front of the user-supplied ones is
almost as good.
This needs a lot more thought, though. Unfortunately, it's making me aware
of some deficiencies in the existing script system, which doesn't allow for
e.g. running scripts with -O at the moment, let alone any other
options. These are allowed in "traditional" distutils scripts, but not
entry-point based scripts, which have no place to specify these things at
the moment. I'm probably going to have to add a way to control these and
other script-generation options at some point.
More information about the Distutils-SIG