[Python-Dev] Finding landmark when prefix != exec-prefix

[Greg Ward]

Hey all --

this is a bug I noticed in 1.5.2 ages ago, and never investigated
further.  I've just figured it out a little bit more; right now I can
only verify it in 1.5, as I don't have the right sort of 1.6 or 2.0
installation at home.  So if this has been fixed, I'll just shut up.

Bottom line: if you have an installation where prefix != exec-prefix,
and there is another Python installation on the system, then Python
screws up finding the landmark file (string.py in 1.5.2) and computes
the wrong prefix and exec-prefix.

Here's the scenario: I have a Red Hat 6.2 installation with the
"official" Red Hat python in /usr/bin/python.  I have a local build
installed with prefix=/usr/local/python and
exec-prefix=/usr/local/python.i86-linux; /usr/local/bin/python is a
symlink to ../python.i86-linux/bin/python.  (This dates to my days of
trying to understand what gets installed where.  Now, of course, I could
tell you what Python installs where in my sleep with one hand tied
behind my back... ;-)

Witness:
  $ /usr/bin/python -c "import sys ; print sys.prefix"
  /usr
  $/usr/local/bin/python -c "import sys ; print sys.prefix"
  /usr

...even though /usr/local/bin/python's library is really in
/usr/local/python/lib/python1.5 and
/usr/local/python.i86-linux/lib/python1.5.

If I erase Red Hat's Python, then /usr/local/bin/python figures out its
prefix correctly.

Using "strace" sheds a little more light on things; here's what I get
after massaging the "strace" output a bit (grep for "string.py"; all
that shows up are 'stat()' calls, where only the last succeeds; I've
stripped out everything but the filename):

  /usr/local/bin/../python.i86-linux/bin/lib/python1.5/string.py
  /usr/local/bin/../python.i86-linux/bin/lib/python1.5/string.pyc
  /usr/local/bin/../python.i86-linux/lib/python1.5/string.py
  /usr/local/bin/../python.i86-linux/lib/python1.5/string.pyc
  /usr/local/bin/../lib/python1.5/string.py
  /usr/local/bin/../lib/python1.5/string.pyc
  /usr/local/bin/lib/python1.5/string.py
  /usr/local/bin/lib/python1.5/string.pyc
  /usr/local/lib/python1.5/string.py
  /usr/local/lib/python1.5/string.pyc
  /usr/lib/python1.5/string.py                # success because of Red Hat's
                                              # Python installation

Well, of course.  Python doesn't know what its true prefix is until it
has found its landmark file, but it can't find its landmark until it
knows its true prefix.  Here's the "strace" output after erasing Red
Hat's Python RPM:

  $ strace /usr/local/bin/python -c 1 2>&1 | grep 'string\.py'
  /usr/local/bin/../python.i86-linux/bin/lib/python1.5/string.py
  /usr/local/bin/../python.i86-linux/bin/lib/python1.5/string.pyc
  /usr/local/bin/../python.i86-linux/lib/python1.5/string.py
  /usr/local/bin/../python.i86-linux/lib/python1.5/string.pyc
  /usr/local/bin/../lib/python1.5/string.py
  /usr/local/bin/../lib/python1.5/string.pyc
  /usr/local/bin/lib/python1.5/string.py
  /usr/local/bin/lib/python1.5/string.pyc
  /usr/local/lib/python1.5/string.py
  /usr/local/lib/python1.5/string.pyc
  /usr/lib/python1.5/string.py               # now fail since I removed 
  /usr/lib/python1.5/string.pyc              # Red Hat's RPM
  /usr/local/python/lib/python1.5/string.py

A-ha!  When the /usr installation is no longer there to fool it, Python
then looks in the right place.

So, has this bug been fixed in 1.6 or 2.0?  If not, where do I look?

        Greg

PS. what about hard-coding a prefix and exec-prefix in the binary, and
only searching for the landmark if the hard-coded values fail?  That
way, this complicated and expensive search is only done if the
installation has been relocated.

-- 
Greg Ward                                      gward@python.net
http://starship.python.net/~gward/