[IPython-dev] Weird PYTHONPATH, etc. issue

[Erik Bray]

On Thu, Sep 27, 2012 at 3:12 PM, David Warde-Farley
<d.warde.farley at gmail.com> wrote:
> On Thu, Sep 27, 2012 at 8:59 AM, Erik Bray <erik.m.bray at gmail.com> wrote:
>
>> I don't really see this so much as a setuptools problem though.  Your
>> sysadmin shouldn't be using PYTHONPATH for this.  I've had to battle
>> this a bit at my own institution.  The problem with using PYTHONPATH
>> to add site-specific site-packages directories are many-fold.  For
>> one, if the user then messes with their PYTHONPATH they suddenly can't
>> `import numpy` anymore and come complaining that numpy broke or
>> something like that.  PYTHONPATH should be there as an easy way for
>> the user to optionally add additional paths to the front of their
>> sys.path.  So when it's used like this it basically takes away your
>> freedom to customize anything on your path at all since the contents
>> of PYTHONPATH come first.  Sure you can always add your stuff to the
>> front of PYTHONPATH too, but a lot of users get confused by that.
>
> Prepending to an environment variable to override things is a pretty
> standard UNIXy thing to do. Same thing appears in Java with CLASSPATH,
> etc., and it pretty much mirrors PATH in that respect. I guess it
> could confuse novices, you're right.

I agree that it's a pretty standard UNIXy thing to do; and that fact
was tugging on the back of my mind as I wrote that. Python, however,
really does not need to rely on PYTHONPATH as much as Java, for
example, relies on CLASSPATH.  And this is a Good Thing.  Python is
designed so that lots of things can be added to its import path
(through sitecustomize.py, usercustomize.py, and other mechanisms)
while keeping PYTHONPATH completely clean, allowing it to be used
easily as a mechanism for run-time user overrides *without* requiring
the user to take great care to keep all the correct items in it, and
carefully restore in when they're done with this.

That's one of the great things about virtualenv--it often surprises
people when I show them virtualenv for the first time that it doesn't
put anything on their PYTHONPATH (although it does modify $PATH).
Relying on PYTHONPATH makes virtualenv less useful because you can't
install something into the virtualenv (a different version of Numpy
for example) and have it override the Numpy imported via the
PYTHONPATH entries.  But I digress.

>> What the sysadmin *should* be doing is adding a sitecustomize.py
>> somewhere on the standard sys.path (in lib/pythonX.Y/site-packages for
>> example) and adding any custom paths there, while leaving your
>> PYTHONPATH alone.  As mentioned elsewhere in this thread
>> usercustomize.py can also be used.
>
> Are you sure this would solve the issue? If all of the stuff in all of
> the site-packages directories is processed before PYTHONPATH is even
> touched, then I guess it might.
>
> David

Generally speaking, yes.  Though I would still believe it if, as
Fernando wrote, setuptools is doing something that even messes
up/overrides PYTHONPATH.  I'm just not sure what that would be as I've
never run into any problems like that.  But I'm not saying they don't
exist, or even denying that they're likely to exist :)

Erik