Mailman 3 August 2014 - pypy-dev

failure installing pygit2 on pypy3 raspbian
by Russ Tremain 10 Aug '14

10 Aug '14

Hi, just wondering if pip install of pygit2 is expected to work, or if this is a problem related to ARM. Also, is there a difference using "pip" vs. pip3 or pip3.2? thanks, -Russ Linux raspberrypi 3.12.22+ #691 PREEMPT Wed Jun 18 18:29:58 BST 2014 armv6l ... pi@raspberrypi:~$ pip3 install pygit2 Downloading/unpacking pygit2 Downloading pygit2-0.21.2.tar.gz (416kB): 416kB downloaded Running setup.py (path:/tmp/pip_build_pi/pygit2/setup.py) egg_info for package pygit2 pygit2/__pycache__/_cffi__gab826d3dx2bbed4fa.c:26:34: error: unknown type name 'git_buf' pygit2/__pycache__/_cffi__gab826d3dx2bbed4fa.c: In function '_cffi_layout__git_buf': pygit2/__pycache__/_cffi__gab826d3dx2bbed4fa.c:35:10: error: unknown type name 'git_buf' pygit2/__pycache__/_cffi__gab826d3dx2bbed4fa.c:37:12: error: 'git_buf' undeclared (first use in this function) pygit2/__pycache__/_cffi__gab826d3dx2bbed4fa.c:1649:21: error: (near initialization for 'nums[13]') [snip - thousands more errors] Traceback (most recent call last): File "/usr/local/pypy3-2.3.1-linux-armhf-raspbian/lib-python/3/distutils/unixccompiler.py", line 131, in _compile extra_postargs) File "/usr/local/pypy3-2.3.1-linux-armhf-raspbian/lib-python/3/distutils/ccompiler.py", line 909, in spawn spawn(cmd, dry_run=self.dry_run) File "/usr/local/pypy3-2.3.1-linux-armhf-raspbian/lib-python/3/distutils/spawn.py", line 32, in spawn _spawn_posix(cmd, search_path, dry_run=dry_run) File "/usr/local/pypy3-2.3.1-linux-armhf-raspbian/lib-python/3/distutils/spawn.py", line 163, in _spawn_posix % (cmd[0], exit_status)) distutils.errors.DistutilsExecError: command 'cc' failed with exit status 1 During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/usr/local/pypy3-2.3.1-linux-armhf-raspbian/lib_pypy/cffi/ffiplatform.py", line 47, in _build dist.run_command('build_ext') File "/usr/local/pypy3-2.3.1-linux-armhf-raspbian/lib-python/3/distutils/dist.py", line 936, in run_command cmd_obj.run() File "/usr/local/pypy3-2.3.1-linux-armhf-raspbian/site-packages/distribute-0.6.49-py3.2.egg/setuptools/command/build_ext.py", line 46, in run _build_ext.run(self) File "/usr/local/pypy3-2.3.1-linux-armhf-raspbian/lib-python/3/distutils/command/build_ext.py", line 354, in run self.build_extensions() File "/usr/local/pypy3-2.3.1-linux-armhf-raspbian/lib-python/3/distutils/command/build_ext.py", line 463, in build_extensions self.build_extension(ext) File "/usr/local/pypy3-2.3.1-linux-armhf-raspbian/site-packages/distribute-0.6.49-py3.2.egg/setuptools/command/build_ext.py", line 182, in build_extension _build_ext.build_extension(self,ext) File "/usr/local/pypy3-2.3.1-linux-armhf-raspbian/lib-python/3/distutils/command/build_ext.py", line 518, in build_extension depends=ext.depends) File "/usr/local/pypy3-2.3.1-linux-armhf-raspbian/lib-python/3/distutils/ccompiler.py", line 574, in compile self._compile(obj, src, ext, cc_args, extra_postargs, pp_opts) File "/usr/local/pypy3-2.3.1-linux-armhf-raspbian/lib-python/3/distutils/unixccompiler.py", line 133, in _compile raise CompileError(msg) distutils.errors.CompileError: command 'cc' failed with exit status 1 During handling of the above exception, another exception occurred: Traceback (most recent call last): File "<string>", line 17, in <module> File "/tmp/pip_build_pi/pygit2/setup.py", line 178, in <module> from ffi import ffi File "pygit2/ffi.py", line 59, in <module> include_dirs=include_dirs, library_dirs=library_dirs) File "/usr/local/pypy3-2.3.1-linux-armhf-raspbian/lib_pypy/cffi/api.py", line 341, in verify lib = self.verifier.load_library() File "/usr/local/pypy3-2.3.1-linux-armhf-raspbian/lib_pypy/cffi/verifier.py", line 74, in load_library self._compile_module() File "/usr/local/pypy3-2.3.1-linux-armhf-raspbian/lib_pypy/cffi/verifier.py", line 139, in _compile_module outputfilename = ffiplatform.compile(tmpdir, self.get_extension()) File "/usr/local/pypy3-2.3.1-linux-armhf-raspbian/lib_pypy/cffi/ffiplatform.py", line 25, in compile outputfilename = _build(tmpdir, ext) File "/usr/local/pypy3-2.3.1-linux-armhf-raspbian/lib_pypy/cffi/ffiplatform.py", line 50, in _build raise VerificationError('%s: %s' % (e.__class__.__name__, e)) cffi.ffiplatform.VerificationError: CompileError: command 'cc' failed with exit status 1 Complete output from command python setup.py egg_info: pygit2/__pycache__/_cffi__gab826d3dx2bbed4fa.c:26:34: error: unknown type name 'git_buf' [snip] ---------------------------------------- Cleaning up... Command python setup.py egg_info failed with error code 1 in /tmp/pip_build_pi/pygit2 Storing debug log for failure in /home/pi/.pip/pip.log pi@raspberrypi:~$

1 0

trouble compiling p4python api with pypy3
by Russ Tremain 10 Aug '14

10 Aug '14

Hi, I'm trying to get p4python (perforce python API) compiling with pypy and I notice that the include headers in pypy are very different than the standard 3.2.5 python headers. Is the C api different for pypy3 from python 3.2.5? For example, note the following diffs: --------------------------------------------------------------------------------------------------- diff -c `find . -name moduleobject.h` *** ./pypy3-2.3.1-linux-armhf-raspbian/include/moduleobject.h 2014-06-19 09:20:58.000000000 -0700 --- ./Python-3.2.5/Include/moduleobject.h 2013-05-15 09:33:41.000000000 -0700 *************** *** 1,3 **** --- 1,4 ---- + /* Module object interface */ #ifndef Py_MODULEOBJECT_H *************** *** 6,11 **** --- 7,30 ---- extern "C" { #endif + PyAPI_DATA(PyTypeObject) PyModule_Type; + + #define PyModule_Check(op) PyObject_TypeCheck(op, &PyModule_Type) + #define PyModule_CheckExact(op) (Py_TYPE(op) == &PyModule_Type) + + PyAPI_FUNC(PyObject *) PyModule_New( + const char *name /* UTF-8 encoded string */ + ); + PyAPI_FUNC(PyObject *) PyModule_GetDict(PyObject *); + PyAPI_FUNC(const char *) PyModule_GetName(PyObject *); + PyAPI_FUNC(const char *) PyModule_GetFilename(PyObject *); + PyAPI_FUNC(PyObject *) PyModule_GetFilenameObject(PyObject *); + #ifndef Py_LIMITED_API + PyAPI_FUNC(void) _PyModule_Clear(PyObject *); + #endif + PyAPI_FUNC(struct PyModuleDef*) PyModule_GetDef(PyObject*); + PyAPI_FUNC(void*) PyModule_GetState(PyObject*); + typedef struct PyModuleDef_Base { PyObject_HEAD PyObject* (*m_init)(void); *************** *** 32,37 **** --- 51,57 ---- freefunc m_free; }PyModuleDef; + #ifdef __cplusplus } #endif --------------------------------------------------------------------------------------------------- When I compile against the distributed pypy3 headers, I get errors: --------------------------------------------------------------------------------------------------- 08/09/14.12:11:29: Building p4python ... 08/09/14.12:11:29: pypy setup.py build --apidir ../p4api-2014.1.886167.main/ --ssl API Release 2014.1 running build running build_py creating build creating build/lib.linux-armv6l-3.2 copying P4.py -> build/lib.linux-armv6l-3.2 running build_ext building 'P4API' extension creating build/temp.linux-armv6l-3.2 cc -O2 -fPIC -Wimplicit -DID_OS="LINUX31ARM" -DID_REL="2012.2" -DID_PATCH="549493" -DID_API="2014.1.main/886167" -DID_Y="2012" -DID_M="11" -DID_D="05" -I../p4api-2014.1.886167.main/ -I../p4api-2014.1.886167.main/include/p4 -I/usr/local/pypy3-2.3.1-linux-armhf-raspbian/include -c P4API.cpp -o build/temp.linux-armv6l-3.2/P4API.o -DOS_LINUX -DOS_LINUX31 -DOS_LINUXARM -DOS_LINUX31ARM cc1plus: warning: command line option '-Wimplicit' is valid for C/ObjC but not for C++ [enabled by default] P4API.cpp: In function 'int P4API_traverse(PyObject*, visitproc, void*)': P4API.cpp:984:5: error: 'PyModule_GetState' was not declared in this scope P4API.cpp: In function 'int P4API_clear(PyObject*)': P4API.cpp:989:5: error: 'PyModule_GetState' was not declared in this scope P4API.cpp: In function 'PyObject* PyInit_P4API()': P4API.cpp:1045:30: error: 'PyModule_GetState' was not declared in this scope error: command 'cc' failed with exit status 1 raspberrypi{gfinstall.5} pypy --version Python 3.2.5 (986752d005bb, Jun 19 2014, 16:20:03) [PyPy 2.3.1 with GCC 4.7.2 20120731 (prerelease)] --------------------------------------------------------------------------------------------------- If I hack around the macro problems, I can get a compile, but then I get a runtime error: --------------------------------------------------------------------------------------------------- raspberrypi{gfinstall.42} cat p4python_version_check.py import P4 print(P4.P4.identify()) raspberrypi{gfinstall.43} pypy p4python_version_check.py Traceback (most recent call last): File "p4python_version_check.py", line 1, in <module> import P4 File "/home/pi/git-fusion/bin/p4python-2012.2.549493/P4.py", line 359, in <module> import P4API ImportError: unable to load extension module '/home/pi/git-fusion/bin/p4python-2012.2.549493/P4API.pypy3-23.so': /home/pi/git-fusion/bin/p4python-2012.2.549493/P4API.pypy3-23.so: undefined symbol: __cxa_pure_virtual --------------------------------------------------------------------------------------------------- Any help appreciated.. tia, -Russ

2 2

Calling lambdas inside loop causes significant slowdown
by Toni Mattis 04 Aug '14

04 Aug '14

Hi, I'm trying to figure out the fastest way in PyPy to introduce abstractions into loops, e.g. refactoring the following code: def sum_direct(data): s = 0 for i in data: if i < 5: s += i + 1 return s to something like: def sum_lambda(data): filter_func = lambda x: x < 5 map_func = lambda x: x + 1 s = 0 for i in data: if filter_func(i): s += map_func(i) return s and then turning both lambdas into arguments, class members and so on. However, the refactoring mentioned above already introduces about 50% of runtime overhead and is not getting better with further refactorings. Shoudn't the tracing/inlining eliminate most of this overhead or is there a mistake on my part? I timed both methods on a large array: from array import array import time data = array('i') for i in xrange(100000000): data.append(i % 10) t = time.time() result = sum_lambda(data) # or sum_direct print result, time.time() - t Calling sum_direct() takes about 0.43 seconds, sum_lambda() is at 0.64s on average. (I'm at changeset 72674:78d5d873a260 from Aug 3 2014, translated and run on Ubuntu 14.04) The JIT trace of the lambda code basically adds two force_token() operations and potentially more expensive guards. Is there any chance to avoid these without excessive metaprogramming? If no, which speedup tricks (speaking of jit hooks, code generation, etc.) can you recommend for implementing such APIs? Thanks in advance, Toni

1 0

cppyy and out arguments
by Martin Matusiak 01 Aug '14

01 Aug '14

Hi, I was reading about cppyy and I could not find a mention of what it does with out arguments. I also read through the unit tests, but I couldn't find any examples. Are they supported? I also found the section header "CPython" a little confusing on this page as the section doesn't seem to talk about cpython: pypy.readthedocs.org/en/latest/cppyy.html#cpython Martin

4 8

libdynd
by Armin Rigo 01 Aug '14

01 Aug '14

Hi, Feedback, from Travis Oliphant at EuroPython: libdynd (https://github.com/ContinuumIO/libdynd) might be the longer-term future of NumPy, and it looks like it would be much more natural to bind to it from PyPy (via cffi). Worth a look I believe. It certainly looks to me like such a cffi binding would be much more user-friendly than numpypy, in the sense that missing functionality would be far easier to contribute back. A bientôt, Armin.

3 4