[Python-Dev] __file__ is not always an absolute path

[Guido van Rossum]

On Sat, Feb 6, 2010 at 12:49 PM, Ezio Melotti <ezio.melotti at gmail.com> wrote:
> In #7712 I was trying to change regrtest to always run the tests in a
> temporary CWD (e.g. /tmp/@test_1234_cwd/).
> The patches attached to the issue add a context manager that changes the
> CWD, and it works fine when I run ./python -m test.regrtest from trunk/.
> However, when I try from trunk/Lib/ it fails with ImportErrors (note that
> the latest patch by Florent Xicluna already tries to workaround the
> problem). The traceback points to "the_package = __import__(abstest,
> globals(), locals(), [])" in runtest_inner (in regrtest.py), and a "print
> __import__('test').__file__" there returns 'test/__init__.pyc'.
> This can be reproduced quite easily:
>
> trunk$ ./python
> Python 2.7a2+ (trunk:77941M, Feb  3 2010, 06:40:49)
> [GCC 4.4.1] on linux2
> Type "help", "copyright", "credits" or "license" for more information.
>>>> import os, sys
>>>> os.getcwd()
> '/home/wolf/dev/trunk'
>>>> import test
>>>> test.__file__  # absolute
> '/home/wolf/dev/trunk/Lib/test/__init__.pyc'
>>>> os.chdir('/tmp')
>>>> test.__file__
> '/home/wolf/dev/trunk/Lib/test/__init__.pyc'
>>>> from test import test_unicode  # works
>>>> test_unicode.__file__
> '/home/wolf/dev/trunk/Lib/test/test_unicode.pyc'
>>>>
> [21]+  Stopped                 ./python
>
> trunk$ cd Lib/
> trunk/Lib$ ../python
> Python 2.7a2+ (trunk:77941M, Feb  3 2010, 06:40:49)
> [GCC 4.4.1] on linux2
> Type "help", "copyright", "credits" or "license" for more information.
>>>> import os, sys
>>>> os.getcwd()
> '/home/wolf/dev/trunk/Lib'
>>>> import test
>>>> test.__file__  # relative
> 'test/__init__.pyc'
>>>> os.chdir('/tmp')
>>>> from test import test_unicode  # fails
> Traceback (most recent call last):
>  File "<stdin>", line 1, in <module>
> ImportError: cannot import name test_unicode
>
> Is there a reason why in the second case test.__file__ is relative?

I haven't tried to repro this particular example, but the reason is
that we don't want to have to call getpwd() on every import nor do we
want to have some kind of in-process variable to cache the current
directory. (getpwd() is relatively slow and can sometimes fail
outright, and trying to cache it has a certain risk of being wrong.)

What we do instead, is code in site.py that walks over the elements of
sys.path and turns them into absolute paths. However this code runs
before '' is inserted in the front of sys.path, so that the initial
value of sys.path is ''.

You may want to print the value of sys.path at various points to see
for yourself.

-- 
--Guido van Rossum (python.org/~guido)