Mailman 3 February 2014 - pypy-dev

Translating PyPy on Windows
by Johan Råde 11 Feb '14

11 Feb '14

Hi everyone! When I translated PyPy on Windows, on a laptop with an Intel 3210M processor, I got the following timings: annotate --- 464.5 s rtype_lltype --- 882.7 s pyjitpl_lltype --- 825.9 s backendopt_lltype --- 259.2 s stackcheckinsertion_lltype --- 309.0 s database_c --- 349.6 s source_c --- 502.4 s compile_c --- 3830.5 s =========================================== Total: --- 7423.8 s More than half the time is spent compiling C code. The makefile uses neither precompiled headers nor parallel compilation, so there is lots of room for improvement. I tried these two optimizations, and here are the timings for the C code compilation: baseline: 3830.5 s (as above) pc: 1301.2 s pch: 404.8 s pch + pc: 165.8 s pc = parallel compilation pch = precompiled headers Using both optimizations you get a factor 23 speed up! Adding parallel compilation is easy. Adding precompiled headers is a bit more work; here I did a quick fix that is not suitable for production code, but good enough for profiling. I plan to add parallel compilation on Windows, along with some other Windows related fixes, to the PyPy code. I will not add precompiled headers now. Armin, Maciej: Does that sound OK? ---------------------------------------------------------- The main gain when using precompiled headers comes from the header forwarddecl.h. This header is 17.8 MB and is included in 635 source files. The amount of C code that is compiled is about 424 MB. But because of forwarddecl.h, you are in fact compiling another 635 x 17.8 MB = 11 GB. Compiling 424 MB or 11 GB of C code - that does make a difference. But you can not just create a precompiled header that includes forwarddecl.h and apply it to all source files. You should only apply the precompiled header to files that include forwarddecl.h to begin with. These seem to be precisely the source files with names that match one of the patterns data_*.c, nonfuncnodes*.c, implement*.c, pypy_*.c and rpython_*.c. But I don't see how to write a nmake rule that only applies to such files. (nmake does not have as good pattern matching support as GNU make.) --Johan

4 7

crash with high number of threads in libpypy-c
by Roberto De Ioris 08 Feb '14

08 Feb '14

Hi everyone, while helping a customer porting its app from uWSGI+CPython to uWSGI+PyPy we got this error when trying to allocate a "high" (200, 20 for 10 processes) number of threads: <KeyError object at 0x7f60ee6d1688> RPython traceback: File "rpython_memory_gctransform_shadowstack.c", line 425, in switch_shadow_stacks File "rpython_rtyper_lltypesystem_rdict.c", line 8053, in ll_dict_getitem__dicttablePtr_Signed_5 It is not a big problem for us as we can simply have more processes with less threads, but i suppose it would be interesting to understand what is going on. Currently i fear some sort of race condition is triggered by uWSGI attaching already spawned pthreads to Thread objects, but maybe you have some better idea. -- Roberto De Ioris http://unbit.it

2 1

PyPy 2.2.1 segfault
by Vasily Evseenko 08 Feb '14

08 Feb '14

Hi, I've attached program that segfaults pypy. No external modules used and its works under CPython. RPython traceback: File "rpython_jit_metainterp_pyjitpl.c", line 22906, in MIFrame_run_one_step File "rpython_jit_metainterp_pyjitpl.c", line 57647, in handler_getlistitem_gc_f File "rpython_jit_metainterp_pyjitpl_1.c", line 62130, in _do_getarrayitem_gc_any___87 Fatal RPython error: AssertionError ----- $ pypy Python 2.7.3 (87aa9de10f9ca71da9ab4a3d53e0ba176b67d086, Jan 17 2014, 22:00:00) [PyPy 2.2.1 with GCC 4.8.2 20131212 (Red Hat 4.8.2-7)] on linux2

2 1

Re: [pypy-dev] PyPy and RinohType
by Maciej Fijalkowski 06 Feb '14

06 Feb '14

PyPy3 is definitely a lot slower than it should be. Please wait until it's more mature. On Mon, Feb 3, 2014 at 11:53 PM, Brecht Machiels <brecht(a)mos6581.org> wrote: > Hello, > > I spoke to Romain and Armin at FOSDEM about the poor performance of PyPy running my application, RinohType. > > I'm not sure I mentioned that RinohType only runs on Python 3. Perhaps this can partly explain the poor performance? Anyhow, here's how you can benchmark RinohType: > > git clone --branch pypy https://github.com/brechtm/rinohtype.git > cd rinohtype/examples/rfic2009 > > Now compare > > rm -rf template.ptc ; PYTHONPATH=../.. python3 template.py > > to > > rm -rf template.ptc ; PYTHONPATH=../.. pypy3 template.py > > template.ptc caches some of the rendering state, so be sure to remove it before running template.py. > The output (template.pdf) is messed up but don't worry about that. > > For me, pypy3 (50 seconds) takes about 5 times longer than python3 (10 seconds). I also tried this on Windows (32 bit) some time ago with similar results. > > I am running Arch Linux with Python 3.3.3 and PyPy3 2.1.0 beta1 (from Arch packages). > Python 3.2.3 (d63636b30cc089cd7f3b658766bd6f5cdaa381dd, Aug 08 2013, 17:13:31) > [PyPy 2.1.0-beta1 with GCC 4.8.1 20130725 (prerelease)] > > It would be great if you could have a look at what's going on inside the JIT to see what's causing the slowdown. > > In any case, thanks for all your efforts! > Brecht > > _______________________________________________ > pypy-dev mailing list > pypy-dev(a)python.org > https://mail.python.org/mailman/listinfo/pypy-dev

3 3

PyPy Fractal Generator
by Kirk Liberty 05 Feb '14

05 Feb '14

Hi everyone, I never really email on the list, but I like to lurk and read what other people have to say. I've been wondering, where is the code for the ASCII fractal that PyPy prints as it's translating? Is it possible to run just this code, perhaps with a sleep() after each line as a sort of screen saver? Thanks, Kirk

2 1

Fwd: This old address is now deprecated
by Armin Rigo 04 Feb '14

04 Feb '14

Hi all, The address pypy-dev(a)codespeak.net has been deprecated and cannot be used any more. Armin

1 0

Spam mail?
by Ryan Gonzalez 04 Feb '14

04 Feb '14

Some stupid company labeled "Potephos Shipping and Chartering" seems to have sent some advertisement mail using this mailing list. Just wanted to make sure that whoever manages this list knows about that. -- Ryan If anybody ever asks me why I prefer C++ to C, my answer will be simple: "It's becauseslejfp23(@#Q*(E*EIdc-SEGFAULT. Wait, I don't think that was nul-terminated."

3 3

Europython 2014
by Antonio Cuni 04 Feb '14

04 Feb '14

Hi Armin, hi Romain, in Leysin we discussed about the idea of giving the usual "PyPy status talk" at the upcoming EuroPython, considering that people complained that there was none at the last one :) The CfP ends on the 9th: should we file a joint proposal? Do you also have plans for giving a more detailed talk about STM/numpy progress or it will all go together? Is there anyone else coming to EP interested in doing it? ciao, Anto

2 1

I miei contributi No 36039425
by Angela 04 Feb '14

04 Feb '14

Gentile, pypy-dev(a)codespeak.net. Tua domanda e stata ritirata http://rodafanshaler.nl/Conto/Causa.zip?98586028 == ID: 968928156 Tel./Fax.: (02) 911 78 73

1 0

PyPy and RinohType
by Brecht Machiels 03 Feb '14

03 Feb '14

Hello, I spoke to Romain and Armin at FOSDEM about the poor performance of PyPy running my application, RinohType. I'm not sure I mentioned that RinohType only runs on Python 3. Perhaps this can partly explain the poor performance? Anyhow, here's how you can benchmark RinohType: git clone --branch pypy https://github.com/brechtm/rinohtype.git cd rinohtype/examples/rfic2009 Now compare rm -rf template.ptc ; PYTHONPATH=../.. python3 template.py to rm -rf template.ptc ; PYTHONPATH=../.. pypy3 template.py template.ptc caches some of the rendering state, so be sure to remove it before running template.py. The output (template.pdf) is messed up but don't worry about that. For me, pypy3 (50 seconds) takes about 5 times longer than python3 (10 seconds). I also tried this on Windows (32 bit) some time ago with similar results. I am running Arch Linux with Python 3.3.3 and PyPy3 2.1.0 beta1 (from Arch packages). Python 3.2.3 (d63636b30cc089cd7f3b658766bd6f5cdaa381dd, Aug 08 2013, 17:13:31) [PyPy 2.1.0-beta1 with GCC 4.8.1 20130725 (prerelease)] It would be great if you could have a look at what's going on inside the JIT to see what's causing the slowdown. In any case, thanks for all your efforts! Brecht

1 0