Mailman 3 June 2015 - pypy-dev

rogue CFFI speedups #....@k....$...#
by anatoly techtonik 23 Jun '15

23 Jun '15

Hi, Just wanted to let you know of awesome 3x speedup after python-tdl was ported to CFFI. python-tdl is a libtcod binding for Python. The wrapped libtcod is the very rad “Doryen Library" used to build roguelikes in C world. I am sure that having it in Python will pave a road to more pleasant text mode interfaces. =) There were some issues during the porting and packaging process. I think you might be interested to check them out: https://github.com/HexDecimal/python-tdl/issues/5 -- anatoly t.

2 1

RFFI Usage and Documentation
by Derek Lockhart 22 Jun '15

22 Jun '15

After a bit of a struggle I finally got RFFI working for a shared libary I had compiled. I found a few strange discrepancies in the behavior of ExternalCompilitionInfo on OSX I thought may benefit from fixes to make behavior more consistent. === RPython Translation vs. Python Execution Strangeness === Given a shared library I had compiled named libsoftfloat.so: - ExternalCompilationInfo( libraries = ['libsoftfloat'] ) and libname == libsoftfloat.so - python execution: Error: "cannot find library libsoftfloat" - rpython translation: Error: "ld: library not found for -llibsoftfloat" - ExternalCompilationInfo( libraries = ['softfloat'] ) and libname == libsoftfloat.so - python execution: Error: "cannot find library softfloat" - rpython translation: Works! The problem with RFFI during Python execution appears to be that on OSX it checks for the library suffix of .dylib, **not** .so: - ExternalCompilationInfo( libraries = ['libsoftfloat'] ) and libname == libsoftfloat.dylib - python execution: Works! - rpython translation: Error: "ld: library not found for -llibsoftfloat" - ExternalCompilationInfo( libraries = ['softfloat'] ) and libname == libsoftfloat.dylib - python execution: Works! - rpython translation: Works! Some naming schemes work on just python execution, some on just rpython translation. It would be nice if these failed/worked consistently in both modes. In particular, I think on OSX the library lookup for RFFI should look for both .so and .dylib, not just .dylib since it seems to be fairly common practice to generate .so files? Also, the libraries naming convention passed to ExternalCompilationInfo is a bit inconsistent/foreign for someone coming from CFFI and using ffi.dlopen(). === Documentation === I really couldn't figure out how to use RFFI from the RPython docs or from the PyPy source. I started off using these resources: - https://rpython.readthedocs.org/en/latest/rffi.html - https://bitbucket.org/pypy/pypy/src/tip/rpython/rlib/_rsocket_rffi.py - https://bitbucket.org/pypy/pypy/src/tip/pypy/module/crypt/interp_crypt.py The only way I really figured out how to use it was from this patch submitted by the libgccjit guy that had some simple examples on how to use RFFI: - https://mail.python.org/pipermail/pypy-dev/2014-December/012947.html What would be really nice is some example of how to write a simple function in C, compile it into a shared library, and use it via RFFI. Even better, an example mapping a CFFI use case to the RFFI interface. I personally learn how to use APIs much more easily with concrete examples; for example, I created a little git repo as a self-demonstration of how to use the new CFFI APIs with a non-trivial library: - https://github.com/dmlockhart/python-softfloat-cffi/ I'd be willing to help a bit with the docs, if it's agreed updates would be appreciated. Derek

3 3

Paper on the HOPE JIT for Python
by Laura Creighton 21 Jun '15

21 Jun '15

http://www.sciencedirect.com/science/article/pii/S2213133714000687 I think some of our poor performance is because they never let the jit warm up, but should we grab some of their benchmarks and see if we can do better? Laura

3 4

Questions about build new extension with RPython
by Boxiang Sun 21 Jun '15

21 Jun '15

Hi all, I am a newbie in PyPy. But with some experience in CPython, and also OpenCV-Python binding and OpenCV-Julia binding.I want to bring OpenCV into PyPy. Then use PyPy to do my research. First, I read some PyPy and RPython document in recent days. I want to wrap OpenCV in PyPy in "low level". Like OpenCV-CPython, it use Python-C API and Numpy C-API to wrap function and class. Then write a autowrap script to wrap it. In this way, OpenCV-CPython can communicate with NumPy array type. If my understanding is correct. NumPy-PyPy wrote in RPython, wrap the corresponding C and Fortran function. I want wrap OpenCV in PyPy just like OpenCV-CPython. Let NumPy ndarray hold the image matrix. But RPython documentation said "It(Write with RPython) cannot be used by any 3rd-party module". Does it mean if wrap OpenCV in PyPy with cffi. OpenCV-PyPy could not use ndarray? I plan to use RPython, but RPython documentation said it need rebuild PyPy it self, and the module write in RPython will become a built-in module. Rebuild PyPy seems unacceptable in OpenCV-PyPy. Is there has a way to wrap OpenCV in PyPy that could communicate with NumPy-PyPy and without rebuild PyPy it self? Please correct me if my understanding was wrong. Regards, Sun

3 3

"ImportError: No module named httplib2"
by Glenn Chin 20 Jun '15

20 Jun '15

On my Ubuntu machine, there's an httplib2 directory under /usr/lib/python2.7/dist-packages . httplib2 imports and works fine in a Python script. However, the above error message appears when I try to run the same script using PyPy 2.6 . I suspect it's a configuration problem. Any suggestions? Glenn

2 1

.whl depending on CFFI
by anatoly techtonik 19 Jun '15

19 Jun '15

Hi, Should binary wheels depend on CFFI? For some reason this seems wrong to me (the package ships with compiled binary modules): >pip install tdl Collecting tdl Downloading tdl-1.3.0-cp27-none-win32.whl (1.6MB) 100% |################################| 1.6MB 50kB/s Collecting cffi>=1.0.0 (from tdl) Downloading cffi-1.1.2-cp27-none-win32.whl (121kB) 100% |################################| 122kB 362kB/s Collecting pycparser (from cffi>=1.0.0->tdl) Downloading pycparser-2.14.tar.gz (223kB) 100% |################################| 225kB 248kB/s Installing collected packages: pycparser, cffi, tdl Running setup.py install for pycparser Build the lexing/parsing tables Successfully installed cffi-1.1.2 pycparser-2.14 tdl-1.3.0 -- anatoly t.

2 1

Scipy
by Miguel Grau 17 Jun '15

17 Jun '15

Dear pypy users, Is pypy working with scipy? I need numpy&scipy for pypy. I installed correctly the nightly PyPy and the NumPy fork but when I try to install scipy I get this error: /usr/include/numpy/npy_math.h:452:27: error: unknown type name ‘npy_clongdouble’ npy_clongdouble npy_csinl(npy_clongdouble z); ^ error: Command "cc -O2 -fPIC -Wimplicit -I/home/user/apps/pypy-c-jit-78122-2c4887192c62-linux64/site-packages/scipy/scipy/special -I/home/user/apps/pypy-c-jit-78122-2c4887192c62-linux64/include -I/home/user/apps/pypy-c-jit-78122-2c4887192c62-linux64/include -I/home/user/apps/pypy-c-jit-78122-2c4887192c62-linux64/site-packages/scipy/scipy/special/c_misc -I/home/user/apps/pypy-c-jit-78122-2c4887192c62-linux64/include -c scipy/special/c_misc/struve.c -o build/temp.linux-x86_64-2.7/scipy/special/c_misc/struve.o" failed with exit status 1 Any idea? Thank you! Miguel

2 1

Heroku PyPy buildpack
by Ram Rachum 16 Jun '15

16 Jun '15

If I want to use a modern version of PyPy on Heroku (they currently only allow PyPy 1.9 by default,) did someone make a Heroku buildpack with a recent PyPy? That'll make it easy for many people to start using PyPy for their web apps.

3 7

pypy 2.6.0 failing with compile errors
by Roshan Cherian 16 Jun '15

16 Jun '15

Hi All, I am building pypy 2.6.0 from source on a oel 6.3 with gcc: -bash-4.1$ gcc --version gcc (GCC) 4.4.6 20120305 (Red Hat 4.4.6-4) I am failing the compile_c task and doing a make: -bash-4.1$ make gcc -O3 -pthread -fomit-frame-pointer -Wall -Wno-unused -I/tmp/pypy260/lib/libffi-3.0.13/include -O3 -g -fPIC -fPIC -fvisibility=hidden -DPy_BUILD_CORE -DPYPY_USE_ASMGCC -DPYPY_JIT_CODEMAP -DPYPY_CPU_HAS_STANDARD_PRECISION -DPy_BUILD_CORE -frandom-seed=testing_1.c -o testing_1.s -S testing_1.c -I"/home/roshan/python/pypy-2.6.0/rpython"/translator/c -I/home/roshan/python/pypy-2.6.0/pypy/module/cpyext/include -I.. -I/home/roshan/python/pypy-2.6.0/pypy/module/_vmprof/src -I/home/roshan/python/pypy-2.6.0/pypy/module/_cffi_backend/src -I/home/roshan/python/pypy-2.6.0/pypy/module/_multibytecodec -I/home/roshan/python/pypy-2.6.0/pypy/module/operator -I/tmp/pypy260/lib/libffi-3.0.13/include In file included from testing_1.c:4: forwarddecl.h:94137: error: expected declaration specifiers or ‘...’ before ‘EC_KEY’ In file included from testing_1.c:4: forwarddecl.h:131517: error: expected ‘=’, ‘,’, ‘;’, ‘asm’ or ‘__attribute__’ before ‘*’ token forwarddecl.h:131518: error: expected ‘)’ before ‘*’ token forwarddecl.h:131520: error: expected declaration specifiers or ‘...’ before ‘EC_KEY’ In file included from testing_1.c:4: forwarddecl.h:155867: error: expected ‘=’, ‘,’, ‘;’, ‘asm’ or ‘__attribute__’ before ‘*’ token make: *** [testing_1.gcmap] Error 1 Could you please help? Thanks, -Roshan

4 9

Re: [pypy-dev] Freezing the ".pypy-26.so" name?
by Laura Creighton 14 Jun '15

14 Jun '15

I think that we had better look at wheels. I only thought it was a better egg, which since I never thought eggs were a good idea, I didn't worry about. But looks like I was wrong. I only started to look at them yesterday, so I am likely full of nonsense but looks like we could make a wheel that doesn't need distutils. Which I still only have found the idea of, and not the code of, which I want to look at. People who find relevant code, please mail me where this is. Laura

1 0