[Distutils] Confusion about the effect of eggs on import

[skip at pobox.com]

(I'm sending this to distutils because I have this vague notion that the
problem might have something to do with setuptools or eggs on sys.path.)

We have a weird issue trying to import matplotlib at work.  Sameer has
matplotlib, numpy and scipy installed in his local directory tree:

    % ls -l /home/titan/sameer/local/lib/python2.4/site-packages 
    total 20
    drwxr-xr-x   5 sameer   research    2560 Feb  9 13:09 IPython
    drwxr-xr-x   7 sameer   research    2560 Feb  9 14:06 matplotlib
    drwxr-xr-x  14 sameer   research    1024 Feb  9 13:02 numpy
    -rw-r--r--   1 sameer   research      31 Dec  9  2004 pylab.py
    -rw-r--r--   1 sameer   research     180 Feb  9 13:46 pylab.pyc
    drwxr-xr-x  22 sameer   research    1024 Feb  9 13:09 scipy

If sys.path doesn't contain any eggs matplotlib is imported from Sameer's
directory.  If it does contain eggs, no matter if they are at the start or
the end, his version of matplotlib isn't found.  Instead, the centrally
installed version is imported.  Sameer's versions of numpy and scipy are
always imported as expected.

Here are three sessions which demonstrate the problem.  In the first session
I explicitly remove the two eggs from sys.path:

    % PYTHONPATH=/home/titan/sameer/local/lib/python2.4/site-packages python
    Python 2.4.2 (#1, Feb 23 2006, 12:48:31)
    [GCC 3.4.1] on sunos5
    Type "help", "copyright", "credits" or "license" for more information.
    >>> import sys.path
    >>> sys.path
    ['/home/titan/skipm/misc/python/python2', '/home/titan/skipm/misc/python', '/opt/app/g++lib6/python-2.4/lib/python2.4/site-packages/3rdParty/gtk-2.6', '', '/opt/app/g++lib6/python-2.4/lib/python2.4/site-packages/setuptools-0.6c3-py2.4.egg', '/opt/app/g++lib6/python-2.4/lib/python2.4/site-packages/SQLAlchemy-0.3.3-py2.4.egg', '/home/titan/sameer/local/lib/python2.4/site-packages', '/opt/app/g++lib6/python-2.4/lib/python2.4', '/opt/app/g++lib6/python-2.4/lib/python2.4/plat-sunos5', '/opt/app/g++lib6/python-2.4/lib/python2.4/lib-tk', '/opt/app/g++lib6/python-2.4/lib/python2.4/lib-dynload', '/opt/app/g++lib6/python-2.4/lib/python2.4/site-packages', '/opt/app/g++lib6/python-2.4/lib/python2.4/site-packages/3rdParty', '/opt/app/g++lib6/python-2.4/lib/python2.4/site-packages/3rdParty/Numeric', '/opt/app/g++lib6/python-2.4/lib/python2.4/site-packages/3rdParty/f2py2e', '/opt/app/g++lib6/python-2.4/lib/python2.4/site-packages/3rdParty/scipy_distutils']
    >>> noneggs = [p for p in sys.path if not p.endswith(".egg")] >>> noneggs
    ['/home/titan/skipm/misc/python/python2', '/home/titan/skipm/misc/python', '/opt/app/g++lib6/python-2.4/lib/python2.4/site-packages/3rdParty/gtk-2.6', '', '/home/titan/sameer/local/lib/python2.4/site-packages', '/opt/app/g++lib6/python-2.4/lib/python2.4', '/opt/app/g++lib6/python-2.4/lib/python2.4/plat-sunos5', '/opt/app/g++lib6/python-2.4/lib/python2.4/lib-tk', '/opt/app/g++lib6/python-2.4/lib/python2.4/lib-dynload', '/opt/app/g++lib6/python-2.4/lib/python2.4/site-packages', '/opt/app/g++lib6/python-2.4/lib/python2.4/site-packages/3rdParty', '/opt/app/g++lib6/python-2.4/lib/python2.4/site-packages/3rdParty/Numeric', '/opt/app/g++lib6/python-2.4/lib/python2.4/site-packages/3rdParty/f2py2e', '/opt/app/g++lib6/python-2.4/lib/python2.4/site-packages/3rdParty/scipy_distutils']
    >>> sys.path = noneggs
    >>> import matplotlib
    >>> matplotlib.__file__
    '/home/titan/sameer/local/lib/python2.4/site-packages/matplotlib/__init__.pyc'

Matplotlib comes from Sameer's directory, as it should.  In the second
session I don't mess with sys.path:

    % PYTHONPATH=/home/titan/sameer/local/lib/python2.4/site-packages python
    Python 2.4.2 (#1, Feb 23 2006, 12:48:31)
    [GCC 3.4.1] on sunos5
    Type "help", "copyright", "credits" or "license" for more information.
    >>> import sys
    >>> sys.path
    ['/home/titan/skipm/misc/python/python2', '/home/titan/skipm/misc/python', '/opt/app/g++lib6/python-2.4/lib/python2.4/site-packages/3rdParty/gtk-2.6', '', '/opt/app/g++lib6/python-2.4/lib/python2.4/site-packages/setuptools-0.6c3-py2.4.egg', '/opt/app/g++lib6/python-2.4/lib/python2.4/site-packages/SQLAlchemy-0.3.3-py2.4.egg', '/home/titan/sameer/local/lib/python2.4/site-packages', '/opt/app/g++lib6/python-2.4/lib/python2.4', '/opt/app/g++lib6/python-2.4/lib/python2.4/plat-sunos5', '/opt/app/g++lib6/python-2.4/lib/python2.4/lib-tk', '/opt/app/g++lib6/python-2.4/lib/python2.4/lib-dynload', '/opt/app/g++lib6/python-2.4/lib/python2.4/site-packages', '/opt/app/g++lib6/python-2.4/lib/python2.4/site-packages/3rdParty', '/opt/app/g++lib6/python-2.4/lib/python2.4/site-packages/3rdParty/Numeric', '/opt/app/g++lib6/python-2.4/lib/python2.4/site-packages/3rdParty/f2py2e', '/opt/app/g++lib6/python-2.4/lib/python2.4/site-packages/3rdParty/scipy_distutils']
    >>> import matplotlib
    Bad key "lines.markeredgecolor" on line 48 in
    /opt/app/g++lib6/python-2.4/lib/python2.4/site-packages/matplotlib/mpl-data/matplotlibrc.
    You probably need to get an updated matplotlibrc file from
    http://matplotlib.sf.net/matplotlibrc or from the matplotlib source
    distribution
    Bad key "lines.markerfacecolor" on line 47 in
    /opt/app/g++lib6/python-2.4/lib/python2.4/site-packages/matplotlib/mpl-data/matplotlibrc.
    You probably need to get an updated matplotlibrc file from
    http://matplotlib.sf.net/matplotlibrc or from the matplotlib source
    distribution
    >>> matplotlib.__file__
    '/opt/app/g++lib6/python-2.4/lib/python2.4/site-packages/matplotlib/__init__.pyc'

Even though Sameer's directory is ahead of the central installation
directory, the centrally installed version is imported.  Here's the final
session.  Rearrange sys.path so all eggs are at the end:

    % PYTHONPATH=/home/titan/sameer/local/lib/python2.4/site-packages python
    Python 2.4.2 (#1, Feb 23 2006, 12:48:31)
    [GCC 3.4.1] on sunos5
    Type "help", "copyright", "credits" or "license" for more information.
    >>> import sys
    >>> sys.path
    ['/home/titan/skipm/misc/python/python2', '/home/titan/skipm/misc/python', '/opt/app/g++lib6/python-2.4/lib/python2.4/site-packages/3rdParty/gtk-2.6', '', '/opt/app/g++lib6/python-2.4/lib/python2.4/site-packages/setuptools-0.6c3-py2.4.egg', '/opt/app/g++lib6/python-2.4/lib/python2.4/site-packages/SQLAlchemy-0.3.3-py2.4.egg', '/home/titan/sameer/local/lib/python2.4/site-packages', '/opt/app/g++lib6/python-2.4/lib/python2.4', '/opt/app/g++lib6/python-2.4/lib/python2.4/plat-sunos5', '/opt/app/g++lib6/python-2.4/lib/python2.4/lib-tk', '/opt/app/g++lib6/python-2.4/lib/python2.4/lib-dynload', '/opt/app/g++lib6/python-2.4/lib/python2.4/site-packages', '/opt/app/g++lib6/python-2.4/lib/python2.4/site-packages/3rdParty', '/opt/app/g++lib6/python-2.4/lib/python2.4/site-packages/3rdParty/Numeric', '/opt/app/g++lib6/python-2.4/lib/python2.4/site-packages/3rdParty/f2py2e', '/opt/app/g++lib6/python-2.4/lib/python2.4/site-packages/3rdParty/scipy_distutils']
    >>> noneggs = [p for p in sys.path if not p.endswith(".egg")] >>> eggs = [p for p in sys.path if p.endswith(".egg")]
    >>> sys.path = noneggs + eggs
    >>> sys.path
    ['/home/titan/skipm/misc/python/python2', '/home/titan/skipm/misc/python', '/opt/app/g++lib6/python-2.4/lib/python2.4/site-packages/3rdParty/gtk-2.6', '', '/home/titan/sameer/local/lib/python2.4/site-packages', '/opt/app/g++lib6/python-2.4/lib/python2.4', '/opt/app/g++lib6/python-2.4/lib/python2.4/plat-sunos5', '/opt/app/g++lib6/python-2.4/lib/python2.4/lib-tk', '/opt/app/g++lib6/python-2.4/lib/python2.4/lib-dynload', '/opt/app/g++lib6/python-2.4/lib/python2.4/site-packages', '/opt/app/g++lib6/python-2.4/lib/python2.4/site-packages/3rdParty', '/opt/app/g++lib6/python-2.4/lib/python2.4/site-packages/3rdParty/Numeric', '/opt/app/g++lib6/python-2.4/lib/python2.4/site-packages/3rdParty/f2py2e', '/opt/app/g++lib6/python-2.4/lib/python2.4/site-packages/3rdParty/scipy_distutils', '/opt/app/g++lib6/python-2.4/lib/python2.4/site-packages/setuptools-0.6c3-py2.4.egg', '/opt/app/g++lib6/python-2.4/lib/python2.4/site-packages/SQLAlchemy-0.3.3-py2.4.egg']
    >>> import matplotlib
    Bad key "lines.markeredgecolor" on line 48 in
    /opt/app/g++lib6/python-2.4/lib/python2.4/site-packages/matplotlib/mpl-data/matplotlibrc.
    You probably need to get an updated matplotlibrc file from
    http://matplotlib.sf.net/matplotlibrc or from the matplotlib source
    distribution
    Bad key "lines.markerfacecolor" on line 47 in
    /opt/app/g++lib6/python-2.4/lib/python2.4/site-packages/matplotlib/mpl-data/matplotlibrc.
    You probably need to get an updated matplotlibrc file from
    http://matplotlib.sf.net/matplotlibrc or from the matplotlib source
    distribution
    >>> matplotlib.__file__
    '/opt/app/g++lib6/python-2.4/lib/python2.4/site-packages/matplotlib/__init__.pyc'

I get the same result if I unset PYTHONSTARTUP.

Any idea what's going on?

Thx,

-- 
Skip Montanaro - skip at pobox.com - http://www.webfast.com/~skip/
"The hippies and the hipsters did some great stuff in the sixties,
but the geeks pulled their weight too." -- Billy Bragg