[Distutils] eggs and PYTHONPATH (cont'd from peak list...)

[Robin Bryce]

> It seems that setuptools redefines the intention of PYTHONPATH from
> being a "runtime path determination" variable to an "installation
> path determination" variable;  why repurpose PYTHONPATH in this way,

I've always seen it like this:
* PYTHONPATH is determined by the shell environment
* .pth files are determined at install time and or (re)configuration time.
* .pth files are merged into PYTHONPATH & sys.path at interpreter startup
* sys.path is runtime determined

And assumed: the only programs that 'casually' do
os.getenv('PYTHONPATH') are shell scripts authored in python rather
than bash. And as such are on PATH.

I missed the start of this thread (on other list) but it looks like
setuptools *has* to get involved in the last three points. It can not
know the results of non egg package startup/runtime sys.path
configuration. It is adding .pth files and manipulating sys.path in
order to do its version dance. setuptools has to pick a precedence for
everything, versioned eggs, versioned setuptools distributions, plain
old python packages found on sys.path.

It looks to me more like setuptools forces version -1.0.0 on
everything that is not an egg, including things identified by a
customised PYTHONPATH.

Phillip said:
> > Eggs always override non-eggs, because eggs can be controlled in
> > ways that non-egg paths can't.

This sounds like a choice not a reason. Isn't it just as reasonable to
say PYTHONPATH can be controlled in ways egg paths can't.

I guess I'm lining up with mike here. On the face of it, it does not
seem reasonable for setuptools to 'guess' the version of a plain
python package on sys.path or otherwise subvert the traditional "use
this package damn it" behaviour of PYTHONPATH. But I don't understand
this stuff enough to state it more strongly than that.

Mike said:
> I would propose the order be this:
>
> eggs on PYTHONPATH
> non-eggs on PYTHONPATH
> eggs in site-packages
> non-eggs in site-packages

<snip>

> However, I cant think of a  possible scenario where a path would explicitly
> exist in PYTHONPATH,  non-egg or egg, where the user would still like the
> system-wide installation to take precedence

I can't either.

mike said:
> For example, if a particular package on a multi-user shell account is
> upgraded systemwide from its previously non-egg version to a
> setuptools/egg version, then any users who have applications running
> on a custom PYTHONPATH setup with dependencies on their locally-
> installed, non-egg versions will now have a broken install; theyd
> have to log in, read all the setuptools docs to figure out whats
> going on, and do the "virtual python" setup in order to fix things

Is this all from the perspective of resolving versions using
setuptools ? If so I think:
- It makes no sense for the system vendor to start using eggs and
*not* also bundle setuptools in this format.
- all non setuptools python packages are effectively *not versioned*
on this system
- The system vendor is making a pretty fundamental change to there
distribution versioning / packaging mechanics in doing this. They
should not be doing this as an interim update to an existing release.
- I always expect my scripts custom config to be broken by distro upgrade.


Are there any python packages widely used by non technical users that
direct their users to perform these sorts of customisations to
PYTHONPATH ?

(I'm assuming egg.link aka develop is an 'egg' package wrt this
discussion and so just works tm)

But I think this issue is distinct from how non setuptools / non
version packages interact with the setuptools world.

Things will look very different when: setuptools becomes part of the
python standard distribution AND system vendors catch up with that
release of python.

Cheers,
Robin