Mailman 3 November 2011 - pypy-dev

Translating pypy-1.7 with -O0
by David Naylor Nov. 24, 2011

Nov. 24, 2011

Hi, I'm trying to translate pypy-1.7 with optimisations at level 0. I get the following error: # /usr/local/bin/pypy translate.py --source --gcrootfinder=shadowstack -- thread -O0 targetpypystandalone.py <snip/> [translation:ERROR] Error: [translation:ERROR] Traceback (most recent call last): [translation:ERROR] File "translate.py", line 308, in main [translation:ERROR] drv.proceed(goals) [translation:ERROR] File "/tmp/home/DragonSA/ports/pypy/work/pypy-pypy- release-1.7/… [View More]pypy/translator/driver.py", line 809, in proceed [translation:ERROR] return self._execute(goals, task_skip = self._maybe_skip()) [translation:ERROR] File "/tmp/home/DragonSA/ports/pypy/work/pypy-pypy- release-1.7/pypy/translator/tool/taskengine.py", line 116, in _execute [translation:ERROR] res = self._do(goal, taskcallable, *args, **kwds) [translation:ERROR] File "/tmp/home/DragonSA/ports/pypy/work/pypy-pypy- release-1.7/pypy/translator/driver.py", line 286, in _do [translation:ERROR] res = func() [translation:ERROR] File "/tmp/home/DragonSA/ports/pypy/work/pypy-pypy- release-1.7/pypy/translator/driver.py", line 505, in task_database_c [translation:ERROR] database = cbuilder.build_database() [translation:ERROR] File "/tmp/home/DragonSA/ports/pypy/work/pypy-pypy- release-1.7/pypy/translator/c/genc.py", line 143, in build_database [translation:ERROR] sandbox=self.config.translation.sandbox) [translation:ERROR] File "/tmp/home/DragonSA/ports/pypy/work/pypy-pypy- release-1.7/pypy/translator/c/database.py", line 63, in __init__ [translation:ERROR] self.gctransformer = self.gcpolicy.transformerclass(translator) [translation:ERROR] File "/tmp/home/DragonSA/ports/pypy/work/pypy-pypy- release-1.7/pypy/rpython/memory/gctransform/framework.py", line 151, in __init__ [translation:ERROR] GCClass, GC_PARAMS = choose_gc_from_config(translator.config) [translation:ERROR] File "/tmp/home/DragonSA/ports/pypy/work/pypy-pypy- release-1.7/pypy/rpython/memory/gc/base.py", line 448, in choose_gc_from_config [translation:ERROR] config.translation.gc,)) [translation:ERROR] ValueError: unknown value for translation.gc: 'ref' Looking at base.py I see that ref and boehm GC are not in the list (and I thought -O0 used the boehm not the ref GC). I saw a previous message about this (re pypy-1.6) but no solution. Also: # file /usr/local/lib/libgc.so.1 /usr/local/lib/libgc.so.1: ELF 64-bit LSB shared object, x86-64, version 1 (FreeBSD), dynamically linked, not stripped confirms I have the required library installed. Is there, perhaps, something I am missing? Regards [View Less]

3 4

Bytecode-less pypy languages
by Timothy Baldridge Nov. 23, 2011

Nov. 23, 2011

So I've started work on my Clojure(lisp) in pypy interpreter again. This time I've decided to take a very materialistic approach to the design. As such my interpreter does not run off of bytecode, instead it directly interprets the Clojure structures, truely treating data as code. The idea is that every object in the system must have a evauate() method, and optionally a invoke(args) method. These methods are implemented as follows: most objects return themselves when evalute() is run def … [View More]

3 2

PyPy 1.7 - widening the sweet spot
by Maciej Fijalkowski Nov. 23, 2011

Nov. 23, 2011

================================== PyPy 1.7 - widening the sweet spot ================================== We're pleased to announce the 1.7 release of PyPy. As became a habit, this release brings a lot of bugfixes and performance improvements over the 1.6 release. However, unlike the previous releases, the focus has been on widening the "sweet spot" of PyPy. That is, classes of Python code that PyPy can greatly speed up should be vastly improved with this release. You can download the 1.7 … [View More]release here: http://pypy.org/download.html What is PyPy? ============= PyPy is a very compliant Python interpreter, almost a drop-in replacement for CPython 2.7. It's fast (`pypy 1.7 and cpython 2.7.1`_ performance comparison) due to its integrated tracing JIT compiler. This release supports x86 machines running Linux 32/64, Mac OS X 32/64 or Windows 32. Windows 64 work is ongoing, but not yet natively supported. The main topic of this release is widening the range of code which PyPy can greatly speed up. On average on our benchmark suite, PyPy 1.7 is around **30%** faster than PyPy 1.6 and up to **20 times** faster on some benchmarks. .. _`pypy 1.7 and cpython 2.7.1`: http://speed.pypy.org Highlights ========== * Numerous performance improvements. There are too many examples which python constructs now should behave faster to list them. * Bugfixes and compatibility fixes with CPython. * Windows fixes. * PyPy now comes with stackless features enabled by default. However, any loop using stackless features will interrupt the JIT for now, so no real performance improvement for stackless-based programs. Contact pypy-dev for info how to help on removing this restriction. * NumPy effort in PyPy was renamed numpypy. In order to try using it, simply write:: import numpypy as numpy at the beginning of your program. There is a huge progress on numpy in PyPy since 1.6, the main feature being implementation of dtypes. * JSON encoder (but not decoder) has been replaced with a new one. This one is written in pure Python, but is known to outperform CPython's C extension up to **2 times** in some cases. It's about **20 times** faster than the one that we had in 1.6. * The memory footprint of some of our RPython modules has been drastically improved. This should impact any applications using for example cryptography, like tornado. * There was some progress in exposing even more CPython C API via cpyext. Things that didn't make it, expect in 1.8 soon ============================================== There is an ongoing work, which while didn't make it to the release, is probably worth mentioning here. This is what you should probably expect in 1.8 some time soon: * Specialized list implementation. There is a branch that implements lists of integers/floats/strings as compactly as array.array. This should drastically improve performance/memory impact of some applications * NumPy effort is progressing forward, with multi-dimensional arrays coming soon. * There are two brand new JIT assembler backends, notably for the PowerPC and ARM processors. Fundraising =========== It's maybe worth mentioning that we're running fundraising campaigns for NumPy effort in PyPy and for Python 3 in PyPy. In case you want to see any of those happen faster, we urge you to donate to `numpy proposal`_ or `py3k proposal`_. In case you want PyPy to progress, but you trust us with the general direction, you can always donate to the `general pot`_. .. _`numpy proposal`: http://pypy.org/numpydonate.html .. _`py3k proposal`: http://pypy.org/py3donate.html .. _`general pot`: http://pypy.org [View Less]

13 24

Translating pypy-sandbox eats 13GB of RAM
by David Naylor Nov. 22, 2011

Nov. 22, 2011

Hi While trying to translate pypy-1.7 with the sandbox option, using pypy-1.6 I saw the translating pypy consume 13.3GB of memory (column Res in top). This is under FreeBSD-9RC2/amd64. I've never noticed this memory consumtion when translating pypy-1.6 with the stackless option. Is this perhaps a bug with pypy-1.6 or the source code for pypy-1.7? I'll try again using pypy-1.7 to do the translating once it has been translated (I needed to restart since I was running the … [View More]

4 4

Re: [pypy-dev] Translating pypy-sandbox eats 13GB of RAM
by Laura Creighton Nov. 21, 2011

Nov. 21, 2011

Are you using gcc 4.2? If so , you have this bug. https://bugs.launchpad.net/ubuntu/+source/gcc-4.2/+bug/187391 and you need a newer gcc. (Or maybe an older one would work, too, but this one doesn't). Laura

2 1

Building pypy on freebsd 64bit
by Shadrin Eugene Nov. 18, 2011

Nov. 18, 2011

Hi! I tryed to build pypy on the freebsd, 64bit platform, but failed. First of all I downloaded a 32bit-Python to run pypy. And then: [user@server_name ~/pypy-trunk/pypy/translator/goal]$ ~/py32/bin/python translate.py -Ojit [translation:info] Translating target as defined by targetpypystandalone [platform:execute] gcc -m32 -c -O3 -pthread -fomit-frame-pointer /var/tmp/usession-trunk-22/gcctest.c -o /var/tmp/usession-trunk-22/gcctest.o [platform:WARNING] /var/tmp/usession-trunk-22/gcctest.… [View More]c:28:2: warning: no newline at end of file [platform:execute] gcc -m32 /var/tmp/usession-trunk-22/gcctest.o -L/lib32 -L/usr/lib32 -L`pwd`/lib32 -Wl,-rpath,/lib32 -Wl,-rpath,/usr/lib32 -pthread -o /var/tmp/usession-trunk-22/gcctest printf("sizeof short=%ld\n", (long)sizeof(short)); printf("sizeof unsigned short=%ld\n", (long)sizeof(unsigned short)); printf("sizeof int=%ld\n", (long)sizeof(int)); printf("sizeof unsigned int=%ld\n", (long)sizeof(unsigned int)); printf("sizeof long=%ld\n", (long)sizeof(long)); printf("sizeof unsigned long=%ld\n", (long)sizeof(unsigned long)); printf("sizeof signed char=%ld\n", (long)sizeof(signed char)); printf("sizeof unsigned char=%ld\n", (long)sizeof(unsigned char)); printf("sizeof long long=%ld\n", (long)sizeof(long long)); printf("sizeof unsigned long long=%ld\n", (long)sizeof(unsigned long long)); printf("sizeof size_t=%ld\n", (long)sizeof(size_t)); printf("sizeof time_t=%ld\n", (long)sizeof(time_t)); printf("sizeof wchar_t=%ld\n", (long)sizeof(wchar_t)); printf("sizeof mode_t=%ld\n", (long)sizeof(mode_t)); printf("sizeof pid_t=%ld\n", (long)sizeof(pid_t)); sizeof short=2 sizeof unsigned short=2 sizeof int=4 sizeof unsigned int=4 sizeof long=8 sizeof unsigned long=8 sizeof signed char=1 sizeof unsigned char=1 sizeof long long=8 sizeof unsigned long long=8 sizeof size_t=8 sizeof time_t=8 sizeof wchar_t=4 sizeof mode_t=2 sizeof pid_t=4 Traceback (most recent call last): File "translate.py", line 300, in <module> main() File "translate.py", line 205, in main targetspec_dic, translateconfig, config, args = parse_options_and_load_target() File "translate.py", line 153, in parse_options_and_load_target targetspec_dic = load_target(targetspec) File "translate.py", line 105, in load_target mod = __import__(specname) File "targetpypystandalone.py", line 5, in <module> from pypy.objspace.std.objspace import StdObjSpace File "/place/home/hitrobot/pypy-trunk/pypy/objspace/std/__init__.py", line 1, in <module> from pypy.objspace.std.objspace import StdObjSpace File "/place/home/hitrobot/pypy-trunk/pypy/objspace/std/objspace.py", line 3, in <module> from pypy.interpreter import pyframe, function, special File "/place/home/hitrobot/pypy-trunk/pypy/interpreter/pyframe.py", line 13, in <module> from pypy.rlib import jit, rstack File "/place/home/hitrobot/pypy-trunk/pypy/rlib/rstack.py", line 10, in <module> from pypy.rpython.lltypesystem import rffi, lltype File "/place/home/hitrobot/pypy-trunk/pypy/rpython/lltypesystem/rffi.py", line 824, in <module> sys.maxint, sizeof(lltype.Signed))) AssertionError: Mixed configuration of the word size of the machine: the underlying Python was compiled with maxint=2147483647, but the C compiler says that 'long' is 8 bytes I got this error, then I added "-m32" flag to gcc argument list (in the posix.py source file) , but error occured again :( Does anybody tryed to build pypy-c on the 64bit FreeBsd? Thank you for your answer! -- Shadrin Eugene [View Less]

5 7

New pypy website
by Maciej Fijalkowski Nov. 18, 2011

Nov. 18, 2011

Hello everyone Ante Salinovic made us a great website for the upcoming 1.7 PyPy release. I would like to see some feedback on it. The preliminary version is now available at http://pypy-site.ep.io Among other things it's missing some more success stories as well as donate buttons for numpy and py3k. Cheers, fijal

6 6

Re: [pypy-dev] Slow sqlite user defined functions with pypy.
by William ML Leslie Nov. 17, 2011

Nov. 17, 2011

Ack. On 17 November 2011 12:23, William ML Leslie <william.leslie.ttg(a)gmail.com> wrote: > On 17 November 2011 12:13, Elefterios Stamatogiannakis <estama(a)gmail.com> wrote: >> Pypy seems to not jit at all when a (pypy) Python function is called from C. > > Calls to native functions must be residualised, as there is no way to > tell what state gives rise to the call to the UDF. > > If there was a loop inside the UDF, that would still be compiled. But … [View More]

4 3

Pure Python Math Library
by Blaine Nov. 17, 2011

Nov. 17, 2011

Does anyone know of a pure python math library? I've been playing around with berp <https://github.com/bjpop/berp/wiki>, which is a python3 to haskell translator and compiler, and it works great as long as you don't go crazy with C extensions. It's highly experimental but fun to play around with. The only thing that I really miss is being able to use the math module. I asked the maintainer if it is possible to map into haskell's math library, but in the mean time a pure python math … [View More]

6 13

Updated 'High Performance Python' tutorial (the one from EuroPython 2011)
by Ian Ozsvald Nov. 16, 2011

Nov. 16, 2011

Dear all, I've published v0.2 of my High Performance Python tutorial write-up from the session I ran at EuroPython: http://ianozsvald.com/2011/07/25/high-performance-python-tutorial-v0-2-from… Antonio - you asked earlier if the 'expanded math' version of the Mandelbrot solver (using doubles rather than complex numbers) would be faster - I've timed it and it is a bit faster with a nightly build of PyPy, but nowhere near as fast at ShedSkin's generated C output (details below). Maciej - thanks … [View More]

5 8