[issue1259] module __file__ attribute shows wrong path
New submission from David <david.schneider@uni-duesseldorf.de>: The __file__ attribute for some modules shows a path corresponding to the system where pypy was translated and not where it is running. This causes one of the module/math app-level tests that use this attribute to load the math_testcases.txt file to fail, if the test are run on a different machine as the one used for translation. Examples:
import math; math.__file__ '/Users/pypy/buildslave/pypy-c-jit-macosx-x86-64/build/pypy/module/math' or import abc; abc.__file__ '/Users/pypy/buildslave/pypy-c-jit-macosx-x86-64/build/lib-python/2.7/abc.py'
In other cases, like ctypes, the behavior is as expected, e.g.:
import ctypes; ctypes.__file__ '/opt/pypy/lib-python/2.7/ctypes/__init__.pyc'
---------- messages: 4736 nosy: bivab, pypy-issue priority: bug status: unread title: module __file__ attribute shows wrong path ________________________________________ PyPy bug tracker <tracker@bugs.pypy.org> <https://bugs.pypy.org/issue1259> ________________________________________
Armin Rigo <armin.rigo@gmail.com> added the comment: "math" should maybe not have a __file__ attribute at all (it doesn't in CPython). "abc" has a broken __file__ attribute probably coming from a .pyc file? But I don't understand why there would be a pyc file in the tarball... ---------- nosy: +arigo status: unread -> chatting ________________________________________ PyPy bug tracker <tracker@bugs.pypy.org> <https://bugs.pypy.org/issue1259> ________________________________________
David <david.schneider@uni-duesseldorf.de> added the comment: For "abc" this also happens when placing the pypy-c binary in a clean checkout and starting it there. ________________________________________ PyPy bug tracker <tracker@bugs.pypy.org> <https://bugs.pypy.org/issue1259> ________________________________________
Amaury Forgeot d Arc <amauryfa@gmail.com> added the comment: "abc.py" is imported before the ObjSpace is frozen; the whole module is probably a prebuilt constant. ---------- nosy: +afa ________________________________________ PyPy bug tracker <tracker@bugs.pypy.org> <https://bugs.pypy.org/issue1259> ________________________________________
bdk <bdkearns@gmail.com> added the comment: $ ./pypy/goal/pypy-c Python 2.7.3 (8556098ab5f0, Feb 10 2013, 04:31:21) [PyPy 2.0.0-beta1 with GCC 4.6.3] on linux2 Type "help", "copyright", "credits" or "license" for more information. And now for something completely different: ``why did you guys have to make the builtin fortune more interesting than actual work? i just catched myself restarting pypy 20 times''
import sys; sys.__file__ '/home/pypy/module/sys' import cPickle; cPickle.__file__ '/home/ubuntu/pypy/lib_pypy/cPickle.pyc' import os; os.__file__ '/home/ubuntu/pypy/lib-python/2.7/os.pyc' import time; time.__file__ '../../pypy/module/rctime'
these are both inconsistent and not correct. can they be fixed to give sane answers? docs seem to imply they should behave like this: http://pypy.readthedocs.org/en/latest/coding-guide.html#determining-the- location-of-a-module-implementation
import sys sys.__file__ '/home/hpk/pypy-dist/pypy/module/sys'
import cPickle cPickle.__file__ '/home/hpk/pypy-dist/lib_pypy/cPickle..py'
import os os.__file__ '/home/hpk/pypy-dist/lib-python/2.7/os.py'
---------- release: -> 2.0 ________________________________________ PyPy bug tracker <tracker@bugs.pypy.org> <https://bugs.pypy.org/issue1259> ________________________________________
Amaury Forgeot d Arc <amauryfa@gmail.com> added the comment: Please explain why they are incorrect. The only weird thing I see is the "../.." in time.__file__. Otherwise they correctly reflect where the implementation lives. ________________________________________ PyPy bug tracker <tracker@bugs.pypy.org> <https://bugs.pypy.org/issue1259> ________________________________________
bdk <bdkearns@gmail.com> added the comment: In my case they should be /home/ubuntu/pypy/pypy/module/sys not /home/pypy/module/sys it would be as if the "hpk/pypy-dist" part of the docs example were always stripped. ________________________________________ PyPy bug tracker <tracker@bugs.pypy.org> <https://bugs.pypy.org/issue1259> ________________________________________
Amaury Forgeot d Arc <amauryfa@gmail.com> added the comment: sys is a builtin-module, so the path is computed at translation time. In my case I see some buildbot working directory. Maybe we should try to not leak this information and hack something like sys.__file__ = "pypy/module/sys" ________________________________________ PyPy bug tracker <tracker@bugs.pypy.org> <https://bugs.pypy.org/issue1259> ________________________________________
bdk <bdkearns@gmail.com> added the comment: right, but both sys and time are builtin modules. what's going on to give them differing attributes? that should be fixed. also to me, the behavior as the docs have it makes the most sense -- at least it matches for a self translated pypy, and for a released build it refers to a nonexistent path but at least parallels the external modules. it's better than pointing off to somewhere that's non even rooted in the pypy dir... ________________________________________ PyPy bug tracker <tracker@bugs.pypy.org> <https://bugs.pypy.org/issue1259> ________________________________________
bdk <bdkearns@gmail.com> added the comment: furthermore, this behavior has changed sometime in the past months: $ ~/pypy-c-jit-57379-5c84521f68f5-linux64/bin/pypy imPython 2.7.3 (5c84521f68f5, Sep 19 2012, 04:35:33) [PyPy 1.9.1-dev0 with GCC 4.7.0] on linux2 Type "help", "copyright", "credits" or "license" for more information. pAnd now for something completely different: ``redefining yellow seems like a better idea''
import sys print sys.__file__ /home/buildslave/bot64/pypy-c-jit-linux-x86-64/build/pypy/module/sys
$ ~/pypy-c-jit-61025-8556098ab5f0-linux64/bin/pypy Python 2.7.3 (8556098ab5f0, Feb 10 2013, 04:31:21) [PyPy 2.0.0-beta1 with GCC 4.6.3] on linux2 Type "help", "copyright", "credits" or "license" for more information. And now for something completely different: ``happy new year''
import sys print sys.__file__ /home/pypy/module/sys
________________________________________ PyPy bug tracker <tracker@bugs.pypy.org> <https://bugs.pypy.org/issue1259> ________________________________________
Amaury Forgeot d Arc <amauryfa@gmail.com> added the comment: - The 1.9.1 version you show was built by the buildbot, and certainly downloaded from nightly builds. - rev 61025 suggest that it's a local clone (with a different history of commits). I guess you compiled it locally, in /home/pypy. The ../.. is a bug IMO, certainly related to the recent rpython split. "The orders have not been changed, said the lamplighter. That is the tragedy." (Le Petit Prince: Chapter 14) ________________________________________ PyPy bug tracker <tracker@bugs.pypy.org> <https://bugs.pypy.org/issue1259> ________________________________________
bdk <bdkearns@gmail.com> added the comment: Both came from the buildbots. Additionally, I have no /home/pypy, and I doubt the buildbot does either. On Mon, Feb 11, 2013 at 1:47 AM, Amaury Forgeot d Arc <tracker@bugs.pypy.org
wrote:
Amaury Forgeot d Arc <amauryfa@gmail.com> added the comment:
- The 1.9.1 version you show was built by the buildbot, and certainly downloaded from nightly builds. - rev 61025 suggest that it's a local clone (with a different history of commits). I guess you compiled it locally, in /home/pypy.
The ../.. is a bug IMO, certainly related to the recent rpython split.
"The orders have not been changed, said the lamplighter. That is the tragedy." (Le Petit Prince: Chapter 14)
________________________________________ PyPy bug tracker <tracker@bugs.pypy.org> <https://bugs.pypy.org/issue1259> ________________________________________
________________________________________ PyPy bug tracker <tracker@bugs.pypy.org> <https://bugs.pypy.org/issue1259> ________________________________________
Amaury Forgeot d Arc <amauryfa@gmail.com> added the comment: Ah, right. It seems that sys.__file__ uses abspath() somewhere. '/home/pypy/module/sys' is '../../pypy/module/sys', relative to the *current* pwd. Bad bad... ________________________________________ PyPy bug tracker <tracker@bugs.pypy.org> <https://bugs.pypy.org/issue1259> ________________________________________
bdk <bdkearns@gmail.com> added the comment: Well, I checked several builtin modules and each I checked was like sys, time is an odd one out, so I'd guess it wasn't something sys was doing specifically but something that time is somehow avoiding. ________________________________________ PyPy bug tracker <tracker@bugs.pypy.org> <https://bugs.pypy.org/issue1259> ________________________________________
Philip Jenvey <pjenvey@underboss.org> added the comment: site.py goes through sys.modules at startup and abspath's all their __file__s, it seems to be the cause (try it w/ -S). I agree these builtin modules shouldn't have a __file__, it's bound to break something ________________________________________ PyPy bug tracker <tracker@bugs.pypy.org> <https://bugs.pypy.org/issue1259> ________________________________________
mattip <matti.picus@gmail.com> added the comment: Is there a reason to keep the __file__ attribute at all? Then help(sys) would need to print, like cpython
help(sys) Help on built-in module sys:
NAME sys FILE (built-in) MODULE DOCS http://docs.python.org/library/sys ... ---------- nosy: +mattip ________________________________________ PyPy bug tracker <tracker@bugs.pypy.org> <https://bugs.pypy.org/issue1259> ________________________________________
participants (6)
-
Amaury Forgeot d Arc -
Armin Rigo
-
bdk
-
David
-
mattip
-
Philip Jenvey