Running Tests for the py3.3 branch
I know I've asked this in irc at least 5 times at this point, but I am still running into issues when I try to run the tests. I have built a pypy binary locally. My exact steps: hg update default cd pypy/goal pypy ../../rpython/bin/rpython --opt=2 #Wait for it to finish... cd .. cd .. PYTHONPATH=. ./pypy-c pypy/tool/build_cffi_imports.py hg update py3.3 pypy/goal/pypy-c pytest.py -sx pypy/module/ (My pypy-c's version info): pypy/goal/pypy-c --version Python 2.7.10 (2cf2803c6652, Dec 17 2015, 06:33:54) [PyPy 4.1.0-alpha0 with GCC 4.8.2] The output of the test line: ==================================================================== test session starts ===================================================================== platform linux2 -- Python 2.7.10[pypy-4.1.0-alpha] -- py-1.4.20 -- pytest-2.5.2 pytest-2.5.2 from /home/lgfdev/Proyectos/my_pypy/pytest.pyc collected 0 items / 1 errors =========================================================================== ERRORS =========================================================================== _____________________________________________________________________ ERROR collecting . _____________________________________________________________________ py/_path/common.py:327: in visit
for x in Visitor(fil, rec, ignore, bf, sort).gen(self):
py/_path/common.py:363: in gen
if p.check(dir=1) and (rec is None or rec(p))])
_pytest/main.py:600: in _recurse
ihook.pytest_collect_directory(path=path, parent=self)
_pytest/main.py:161: in call_matching_hooks
plugins = self.config._getmatchingplugins(self.fspath)
_pytest/config.py:670: in _getmatchingplugins
plugins += self._conftest.getconftestmodules(fspath)
_pytest/config.py:512: in getconftestmodules
clist.append(self.importconftest(conftestpath))
_pytest/config.py:538: in importconftest
self._conftestpath2mod[conftestpath] = mod =
conftestpath.pyimport() py/_path/local.py:620: in pyimport
__import__(modname)
class Module(MixedModule):
pypy/module/select/__init__.py:8: in <module> pypy/module/select/__init__.py:22: in Module
from pypy.module.select.interp_epoll import public_symbols
from pypy.interpreter.typedef import TypeDef, GetSetProperty
@interp2app
pypy/module/select/interp_epoll.py:9: in <module> pypy/interpreter/typedef.py:66: in <module> pypy/interpreter/gateway.py:929: in __new__
doc=doc)
pypy/interpreter/gateway.py:598: in __init__
from pypy.interpreter import pycode
from pypy.tool.stdlib_opcode import opcodedesc, HAVE_ARGUMENT
load_pypy_opcode()
pypy/interpreter/pycode.py:17: in <module> pypy/tool/stdlib_opcode.py:31: in <module> pypy/tool/stdlib_opcode.py:22: in load_pypy_opcode
from pypy.tool.lib_pypy import LIB_PYTHON
from pypy.module.sys.version import CPYTHON_VERSION
from rpython.rlib import compilerinfo rpython/rlib/compilerinfo.py:2: in <module> from rpython.rtyper.lltypesystem import rffi rpython/rtyper/lltypesystem/rffi.py:5: in <module> from rpython.rtyper.lltypesystem import ll2ctypes rpython/rtyper/lltypesystem/ll2ctypes.py:4: in <module> import ctypes
from _ctypes import Union, Structure, Array
from _ctypes.basics import _CData, sizeof, alignment, byref, addressof,\ E File "/home/lgfdev/Proyectos/my_pypy/lib_pypy/_ctypes/basics.py",
pypy/tool/lib_pypy.py:4: in <module> pypy/module/sys/version.py:5: in <module> lib-python/2.7/ctypes/__init__.py:11: in <module> lib_pypy/_ctypes/__init__.py:2: in <module> line 138 E class _CData(object, metaclass=_CDataMeta): E ^ E SyntaxError: invalid syntax ================================================================== 1 error in 3.36 seconds =================================================================== What have I screwed up? Interestingly to me, if I run the tests using the apt-get-provided pypy (I think this one came from http://ppa.launchpad.net/pypy/ppa/ubuntu/, but I could be misremembering), it fails differently: pypy --version Python 2.7.10 (4.0.1+dfsg-1~ppa1~ubuntu14.04, Nov 20 2015, 19:34:15) [PyPy 4.0.1 with GCC 4.8.4] pypy pytest.py -sx pypy/module =================================================================================================================================================== test session starts ==================================================================================================================================================== platform linux2 -- Python 2.7.10[pypy-4.0.1-final] -- py-1.4.20 -- pytest-2.5.2 pytest-2.5.2 from /home/lgfdev/Proyectos/my_pypy/pytest.py [platform:execute] gcc -c -O3 -pthread -fomit-frame-pointer -Wall -Wno-unused /tmp/usession-py3.3-228/platcheck_10.c -o /tmp/usession-py3.3-228/platcheck_10.o [platform:execute] gcc -c -O3 -pthread -fomit-frame-pointer -Wall -Wno-unused /tmp/usession-py3.3-228/platcheck_15.c -o /tmp/usession-py3.3-228/platcheck_15.o [platform:execute] gcc -c -O3 -pthread -fomit-frame-pointer -Wall -Wno-unused /tmp/usession-py3.3-228/platcheck_18.c -o /tmp/usession-py3.3-228/platcheck_18.o [platform:execute] gcc -c -O3 -pthread -fomit-frame-pointer -Wall -Wno-unused /tmp/usession-py3.3-228/platcheck_19.c -o /tmp/usession-py3.3-228/platcheck_19.o [platform:execute] gcc -c -O3 -pthread -fomit-frame-pointer -Wall -Wno-unused /tmp/usession-py3.3-228/platcheck_20.c -o /tmp/usession-py3.3-228/platcheck_20.o [platform:execute] gcc -c -O3 -pthread -fomit-frame-pointer -Wall -Wno-unused /tmp/usession-py3.3-228/platcheck_25.c -o /tmp/usession-py3.3-228/platcheck_25.o [platform:execute] gcc -c -O3 -pthread -fomit-frame-pointer -Wall -Wno-unused /tmp/usession-py3.3-228/platcheck_36.c -o /tmp/usession-py3.3-228/platcheck_36.o [platform:execute] gcc -c -O3 -pthread -fomit-frame-pointer -Wall -Wno-unused /tmp/usession-py3.3-228/platcheck_38.c -o /tmp/usession-py3.3-228/platcheck_38.o [platform:execute] gcc -c -O3 -pthread -fomit-frame-pointer -Wall -Wno-unused -fPIC -fvisibility=hidden -DRPY_EXTERN=RPY_EXPORTED -I/home/lgfdev/Proyectos/my_pypy/pypy/module/cppyy/include -I/home/lgfdev/Proyectos/my_pypy/pypy/module/cppyy/test -I/home/lgfdev/Proyectos/my_pypy/rpython/translator/c /home/lgfdev/Proyectos/my_pypy/pypy/module/cppyy/src/dummy_backend.cxx -o /tmp/usession-py3.3-228/pypy/module/cppyy/src/dummy_backend.o [platform:execute] g++ -shared /tmp/usession-py3.3-228/pypy/module/cppyy/src/dummy_backend.o -pthread -Wl,--export-dynamic -lrt -o /tmp/usession-py3.3-228/pypy/module/cppyy/src/libcppyy_dummy_backend.so [platform:execute] gcc -c -O3 -pthread -fomit-frame-pointer -Wall -Wno-unused /tmp/usession-py3.3-228/platcheck_50.c -o /tmp/usession-py3.3-228/platcheck_50.o [platform:execute] gcc /tmp/usession-py3.3-228/platcheck_50.o -pthread -Wl,--export-dynamic -lintl -lrt -o /tmp/usession-py3.3-228/platcheck_50 [platform:execute] gcc -c -O3 -pthread -fomit-frame-pointer -Wall -Wno-unused /tmp/usession-py3.3-228/platcheck_61.c -o /tmp/usession-py3.3-228/platcheck_61.o [platform:execute] gcc -c -O3 -pthread -fomit-frame-pointer -Wall -Wno-unused -DPy_BUILD_CORE -I/home/lgfdev/Proyectos/my_pypy/pypy/module/cpyext/include -I/home/lgfdev/Proyectos/my_pypy/rpython/translator/c -I/tmp/usession-py3.3-228 /tmp/usession-py3.3-228/platcheck_62.c -o /tmp/usession-py3.3-228/platcheck_62.o [platform:execute] gcc /tmp/usession-py3.3-228/platcheck_62.o -pthread -Wl,--export-dynamic -lrt -o /tmp/usession-py3.3-228/platcheck_62 [platform:execute] gcc -c -O3 -pthread -fomit-frame-pointer -Wall -Wno-unused -DPy_BUILD_CORE -I/home/lgfdev/Proyectos/my_pypy/pypy/module/cpyext/include -I/home/lgfdev/Proyectos/my_pypy/rpython/translator/c -I/tmp/usession-py3.3-228 /tmp/usession-py3.3-228/platcheck_63.c -o /tmp/usession-py3.3-228/platcheck_63.o [platform:execute] gcc /tmp/usession-py3.3-228/platcheck_63.o -pthread -Wl,--export-dynamic -lrt -o /tmp/usession-py3.3-228/platcheck_63 [platform:execute] gcc -c -O3 -pthread -fomit-frame-pointer -Wall -Wno-unused -DPy_BUILD_CORE -I/home/lgfdev/Proyectos/my_pypy/pypy/module/cpyext/include -I/home/lgfdev/Proyectos/my_pypy/rpython/translator/c -I/tmp/usession-py3.3-228 /tmp/usession-py3.3-228/platcheck_64.c -o /tmp/usession-py3.3-228/platcheck_64.o [platform:execute] gcc /tmp/usession-py3.3-228/platcheck_64.o -pthread -Wl,--export-dynamic -lrt -o /tmp/usession-py3.3-228/platcheck_64 [platform:execute] gcc -c -O3 -pthread -fomit-frame-pointer -Wall -Wno-unused -fPIC -fvisibility=hidden -g -O0 -DRPY_EXTERN=RPY_EXPORTED -DRPYTHON_LL2CTYPES -I/home/lgfdev/Proyectos/my_pypy/pypy/module/_codecs -I/home/lgfdev/Proyectos/my_pypy/rpython/translator/c /home/lgfdev/Proyectos/my_pypy/pypy/module/_codecs/locale_codec.c -o /tmp/usession-py3.3-228/pypy/module/_codecs/locale_codec.o [platform:execute] gcc -shared /tmp/usession-py3.3-228/pypy/module/_codecs/locale_codec.o -pthread -Wl,--export-dynamic -lrt -o /tmp/usession-py3.3-228/shared_cache/externmod.so [platform:execute] gcc -c -O3 -pthread -fomit-frame-pointer -Wall -Wno-unused -fPIC -fvisibility=hidden -g -O0 -DRPY_EXTERN=RPY_EXPORTED -DRPYTHON_LL2CTYPES /tmp/usession-py3.3-228/module_cache/module_0.c -o /tmp/usession-py3.3-228/module_cache/module_0.o [platform:execute] gcc -c -O3 -pthread -fomit-frame-pointer -Wall -Wno-unused -fPIC -fvisibility=hidden -g -O0 -DRPY_EXTERN=RPY_EXPORTED -DRPYTHON_LL2CTYPES /tmp/usession-py3.3-228/module_cache/module_1.c -o /tmp/usession-py3.3-228/module_cache/module_1.o [platform:execute] gcc -c -O3 -pthread -fomit-frame-pointer -Wall -Wno-unused -fPIC -fvisibility=hidden -g -O0 -DRPY_EXTERN=RPY_EXPORTED -DRPYTHON_LL2CTYPES /tmp/usession-py3.3-228/module_cache/module_2.c -o /tmp/usession-py3.3-228/module_cache/module_2.o [platform:execute] gcc -shared /tmp/usession-py3.3-228/module_cache/module_0.o /tmp/usession-py3.3-228/module_cache/module_1.o /tmp/usession-py3.3-228/module_cache/module_2.o -pthread -Wl,--export-dynamic -lrt -o /tmp/usession-py3.3-228/shared_cache/externmod_0.so [platform:execute] gcc -c -O3 -pthread -fomit-frame-pointer -Wall -Wno-unused -fPIC -fvisibility=hidden -g -O0 -DRPY_EXTERN=RPY_EXPORTED -DRPYTHON_LL2CTYPES /tmp/usession-py3.3-228/module_cache/module_3.c -o /tmp/usession-py3.3-228/module_cache/module_3.o [platform:execute] gcc -shared /tmp/usession-py3.3-228/module_cache/module_3.o -pthread -Wl,--export-dynamic -lrt -o /tmp/usession-py3.3-228/shared_cache/externmod_1.so [platform:execute] gcc -c -O3 -pthread -fomit-frame-pointer -Wall -Wno-unused -fPIC -fvisibility=hidden -g -O0 -DRPY_EXTERN=RPY_EXPORTED -DRPYTHON_LL2CTYPES /tmp/usession-py3.3-228/module_cache/module_4.c -o /tmp/usession-py3.3-228/module_cache/module_4.o [platform:execute] gcc -c -O3 -pthread -fomit-frame-pointer -Wall -Wno-unused -fPIC -fvisibility=hidden -g -O0 -DRPY_EXTERN=RPY_EXPORTED -DRPYTHON_LL2CTYPES /tmp/usession-py3.3-228/module_cache/module_5.c -o /tmp/usession-py3.3-228/module_cache/module_5.o [platform:execute] gcc -c -O3 -pthread -fomit-frame-pointer -Wall -Wno-unused -fPIC -fvisibility=hidden -g -O0 -DRPY_EXTERN=RPY_EXPORTED -DRPYTHON_LL2CTYPES /tmp/usession-py3.3-228/module_cache/module_6.c -o /tmp/usession-py3.3-228/module_cache/module_6.o [platform:execute] gcc -c -O3 -pthread -fomit-frame-pointer -Wall -Wno-unused -fPIC -fvisibility=hidden -g -O0 -DRPY_EXTERN=RPY_EXPORTED -DRPYTHON_LL2CTYPES /tmp/usession-py3.3-228/module_cache/module_7.c -o /tmp/usession-py3.3-228/module_cache/module_7.o [platform:execute] gcc -c -O3 -pthread -fomit-frame-pointer -Wall -Wno-unused -fPIC -fvisibility=hidden -g -O0 -DRPY_EXTERN=RPY_EXPORTED -DRPYTHON_LL2CTYPES /tmp/usession-py3.3-228/module_cache/module_8.c -o /tmp/usession-py3.3-228/module_cache/module_8.o [platform:execute] gcc -c -O3 -pthread -fomit-frame-pointer -Wall -Wno-unused -fPIC -fvisibility=hidden -g -O0 -DRPY_EXTERN=RPY_EXPORTED -DRPYTHON_LL2CTYPES /tmp/usession-py3.3-228/module_cache/module_9.c -o /tmp/usession-py3.3-228/module_cache/module_9.o [platform:execute] gcc -c -O3 -pthread -fomit-frame-pointer -Wall -Wno-unused -fPIC -fvisibility=hidden -g -O0 -DRPY_EXTERN=RPY_EXPORTED -DRPYTHON_LL2CTYPES /tmp/usession-py3.3-228/module_cache/module_10.c -o /tmp/usession-py3.3-228/module_cache/module_10.o [platform:execute] gcc -c -O3 -pthread -fomit-frame-pointer -Wall -Wno-unused -fPIC -fvisibility=hidden -g -O0 -DRPY_EXTERN=RPY_EXPORTED -DRPYTHON_LL2CTYPES /tmp/usession-py3.3-228/module_cache/module_11.c -o /tmp/usession-py3.3-228/module_cache/module_11.o [platform:execute] gcc -c -O3 -pthread -fomit-frame-pointer -Wall -Wno-unused -fPIC -fvisibility=hidden -g -O0 -DRPY_EXTERN=RPY_EXPORTED -DRPYTHON_LL2CTYPES /tmp/usession-py3.3-228/module_cache/module_12.c -o /tmp/usession-py3.3-228/module_cache/module_12.o [platform:execute] gcc -c -O3 -pthread -fomit-frame-pointer -Wall -Wno-unused -fPIC -fvisibility=hidden -g -O0 -DRPY_EXTERN=RPY_EXPORTED -DRPYTHON_LL2CTYPES /tmp/usession-py3.3-228/module_cache/module_13.c -o /tmp/usession-py3.3-228/module_cache/module_13.o [platform:execute] gcc -c -O3 -pthread -fomit-frame-pointer -Wall -Wno-unused -fPIC -fvisibility=hidden -g -O0 -DRPY_EXTERN=RPY_EXPORTED -DRPYTHON_LL2CTYPES /tmp/usession-py3.3-228/module_cache/module_14.c -o /tmp/usession-py3.3-228/module_cache/module_14.o [platform:execute] gcc -c -O3 -pthread -fomit-frame-pointer -Wall -Wno-unused -fPIC -fvisibility=hidden -g -O0 -DRPY_EXTERN=RPY_EXPORTED -DRPYTHON_LL2CTYPES /tmp/usession-py3.3-228/module_cache/module_15.c -o /tmp/usession-py3.3-228/module_cache/module_15.o [platform:WARNING] /tmp/usession-py3.3-228/module_cache/module_15.c: In function ‘pypy_macro_wrapper_mknod’: [platform:WARNING] /tmp/usession-py3.3-228/module_cache/module_15.c:88:1: warning: implicit declaration of function ‘mknod’ [-Wimplicit-function-declaration] [platform:WARNING] RPY_EXTERN int pypy_macro_wrapper_mknod(char *arg0, int arg1, int arg2) { return mknod(arg0, arg1, arg2); } [platform:WARNING] ^ [platform:execute] gcc -shared /tmp/usession-py3.3-228/module_cache/module_4.o /tmp/usession-py3.3-228/module_cache/module_5.o /tmp/usession-py3.3-228/module_cache/module_6.o /tmp/usession-py3.3-228/module_cache/module_7.o /tmp/usession-py3.3-228/module_cache/module_8.o /tmp/usession-py3.3-228/module_cache/module_9.o /tmp/usession-py3.3-228/module_cache/module_10.o /tmp/usession-py3.3-228/module_cache/module_11.o /tmp/usession-py3.3-228/module_cache/module_12.o /tmp/usession-py3.3-228/module_cache/module_13.o /tmp/usession-py3.3-228/module_cache/module_14.o /tmp/usession-py3.3-228/module_cache/module_15.o -pthread -Wl,--export-dynamic -lutil -lrt -o /tmp/usession-py3.3-228/shared_cache/externmod_2.so [platform:execute] gcc -c -O3 -pthread -fomit-frame-pointer -Wall -Wno-unused -fPIC -fvisibility=hidden -I/home/lgfdev/Proyectos/my_pypy/rpython/translator/c /home/lgfdev/Proyectos/my_pypy/pypy/module/test_lib_pypy/ctypes_tests/_ctypes_test.c -o /tmp/usession-py3.3-228/pypy/module/test_lib_pypy/ctypes_tests/_ctypes_test.o [platform:WARNING] /home/lgfdev/Proyectos/my_pypy/pypy/module/test_lib_pypy/ctypes_tests/_ctypes_test.c:256:14: warning: ‘an_integer’ initialized and declared ‘extern’ [enabled by default] [platform:WARNING] EXPORT (int) an_integer = 42; [platform:WARNING] ^ [platform:WARNING] /home/lgfdev/Proyectos/my_pypy/pypy/module/test_lib_pypy/ctypes_tests/_ctypes_test.c:263:14: warning: ‘a_string’ initialized and declared ‘extern’ [enabled by default] [platform:WARNING] EXPORT(char) a_string[16] = "0123456789abcdef"; [platform:WARNING] ^ [platform:WARNING] /home/lgfdev/Proyectos/my_pypy/pypy/module/test_lib_pypy/ctypes_tests/_ctypes_test.c:323:19: warning: ‘last_tf_arg_s’ initialized and declared ‘extern’ [enabled by default] [platform:WARNING] EXPORT(LONG_LONG) last_tf_arg_s = 0; [platform:WARNING] ^ [platform:WARNING] /home/lgfdev/Proyectos/my_pypy/pypy/module/test_lib_pypy/ctypes_tests/_ctypes_test.c:324:28: warning: ‘last_tf_arg_u’ initialized and declared ‘extern’ [enabled by default] [platform:WARNING] EXPORT(unsigned LONG_LONG) last_tf_arg_u = 0; [platform:WARNING] ^ [platform:execute] gcc -shared /tmp/usession-py3.3-228/pypy/module/test_lib_pypy/ctypes_tests/_ctypes_test.o -pthread -Wl,--export-dynamic -lrt -o /tmp/pytest-33/_ctypes_test/_ctypes_test.so collecting 1197 items / 1 errors ========================================================================================================================================================== ERRORS ========================================================================================================================================================== _________________________________________________________________________________________________________________________ ERROR collecting pypy/module/_posixsubprocess/test/test_ztranslation.py __________________________________________________________________________________________________________________________ import file mismatch: imported module 'test_ztranslation' has this __file__ attribute: /home/lgfdev/Proyectos/my_pypy/pypy/module/_hashlib/test/test_ztranslation.py which is not the same as the test file we want to collect: /home/lgfdev/Proyectos/my_pypy/pypy/module/_posixsubprocess/test/test_ztranslation.py HINT: remove __pycache__ / .pyc files and/or use a unique basename for your test file modules !!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!! Interrupted: stopping after 1 failures !!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!! ================================================================================================================================================= 1 error in 50.85 seconds ================================================================================================================================================= This test is not failing for the regular py3.3 buildbot. Similarly, if I run specific files which are currently failing for the py3.3 buildbot, they all just work on my machine when tested using the "pypy" executable. (A couple have actually failed the same as they have failed for the py3.3 buildbot, but every still-failing test that I have specifically run passes locally now, so something must be wrong.) Why is there a difference in behavior between my /usr/bin/pypy and my newly-built pypy? Why are neither of the pytest.py commands matching the output of the tests run by the buildbot? I don't really understand how lib_pypy is supposed to interact with the python used to invoke the tests. The behavior seen when I run the tests with my pypy-c makes sense to me, failing as soon as it runs into python3-specific code. I don't really get why the tests don't fail the same way when I run the tests using the "pypy" executable. If anyone could explain where I have gone wrong, it would be greatly, greatly appreciated. Thanks
The pypy/goal/pypy-c output: http://pastebin.com/b5MQPPH8 "pypy" output: http://pastebin.com/2kBPLKjy "python2" output: http://pastebin.com/hpWa24zZ
After talking to mjacob, I now understand why the pypy-c tests were failing so badly. Re the pypy/python2 errors: It looks like there might be some bug in pytest. If you try running this command: pypy pytest.py pypy/module/_hashlib/ pypy/module/faulthandler/ It should fail with the same error seen in the python2 pastebin above. I now understand that the buildbot must not run these two tests in the same test execution, explaining why the behavior I'm seeing is different. Given all this, I think I now understand this correctly. Thanks again to mjacob for explaining!
In case anyone is curious: The pypy-c tests were failing because ultimately, the pypy binary has to always use lib_pypy. When I updated my working copy to py3.3 after generating the pypy-c binary, my pypy-c binary, which was still just a python2 binary, was now incompatible with the working copy's lib_pypy, which expected a python 3.3 binary. Using my apt-get provided pypy works properly because it is using its own lib_pypy defined elsewhere (which is still correctly a python2-compatible lib_pypy, not my now-3.3 lib_pypy). And of course, using "python2" works fine because it doesn't use any lib_pypy at all.
participants (1)
-
marky1991 .