Mailman 3 October 2011 - pypy-dev

Python vs pypy: interesting performance difference [dict.setdefault]
by David Naylor 10 Mar '14

10 Mar '14

Hi, I needed to create a cache of date and time objects and I wondered what was the best way to handle the cache. For comparison I put together the following test: <file="iforkey.py"> import datetime import random import timeit ranges = [datetime.datetime(2011,01, random.randint(1, 31)) for i in xrange(1000)] def ifdict(): cache = {} b = [] for i in ranges: key = i.day if key in cache: b.append(cache[key]) else: date = i.date() cache[key] = date b.append(date) def keydict(): cache = {} b = [] for i in ranges: key = i.day try: b.append(cache[key]) except KeyError: date = i.date() cache[key] = date b.append(date) def defaultdict(): cache = {} b= [] for i in ranges: b.append(cache.setdefault(i, i.date())) print "ifdict:", timeit.repeat("ifdict()", "from __main__ import ifdict", number=10000) print "keydict:", timeit.repeat("keydict()", "from __main__ import keydict", number=10000) print "defaultdict:", timeit.repeat("defaultdict()", "from __main__ import defaultdict", number=10000) </file> # python iforfile.py ifdict: [2.432887077331543, 2.4002890586853027, 2.397233009338379] keydict: [2.3483030796051025, 2.358638048171997, 2.314802885055542] defaultdict: [3.5384328365325928, 3.5329859256744385, 3.5728111267089844] # pypy iforfile.py (pypy 1.5) ifdict: [0.8129069805145264, 0.74648118019104, 0.7432689666748047] keydict: [0.5187451839447021, 0.4662129878997803, 0.4504108428955078] defaultdict: [37.98510789871216, 37.859113931655884, 37.92770600318909] Pypy displays significant slowdown in the defaultdict function, otherwise displays its usual speedup. To check what is the cause I replaced i.date() with i.day and found no major difference in times. It appears dict.setdefault (or it's interaction with jit) is causing a slow down. Regards

4 14

PyPy 1.6 not working on Windows XP
by Ram Rachum 05 Dec '11

05 Dec '11

Hey guys, I upgraded to PyPy 1.6 on my 2 Windows XP 32 bit machines. It crashes on both system when running the GarlicSim test suite. It shows a Windows error dialog saying "pypy.exe has encountered a problem and needs to close. We are sorry for the inconvenience." and giving this data: AppName: pypy.exe AppVer: 0.0.0.0 ModName: kernel32.dll ModVer: 5.1.2600.5512 Offset: 00040b0d I can also open a dialog with a lot of data on the error (don't know how useful it is) but Windows won't let me Ctrl-C it. What can I do? Thanks, Ram.

8 32

Building pypy on freebsd 64bit
by Shadrin Eugene 18 Nov '11

18 Nov '11

Hi! I tryed to build pypy on the freebsd, 64bit platform, but failed. First of all I downloaded a 32bit-Python to run pypy. And then: [user@server_name ~/pypy-trunk/pypy/translator/goal]$ ~/py32/bin/python translate.py -Ojit [translation:info] Translating target as defined by targetpypystandalone [platform:execute] gcc -m32 -c -O3 -pthread -fomit-frame-pointer /var/tmp/usession-trunk-22/gcctest.c -o /var/tmp/usession-trunk-22/gcctest.o [platform:WARNING] /var/tmp/usession-trunk-22/gcctest.c:28:2: warning: no newline at end of file [platform:execute] gcc -m32 /var/tmp/usession-trunk-22/gcctest.o -L/lib32 -L/usr/lib32 -L`pwd`/lib32 -Wl,-rpath,/lib32 -Wl,-rpath,/usr/lib32 -pthread -o /var/tmp/usession-trunk-22/gcctest printf("sizeof short=%ld\n", (long)sizeof(short)); printf("sizeof unsigned short=%ld\n", (long)sizeof(unsigned short)); printf("sizeof int=%ld\n", (long)sizeof(int)); printf("sizeof unsigned int=%ld\n", (long)sizeof(unsigned int)); printf("sizeof long=%ld\n", (long)sizeof(long)); printf("sizeof unsigned long=%ld\n", (long)sizeof(unsigned long)); printf("sizeof signed char=%ld\n", (long)sizeof(signed char)); printf("sizeof unsigned char=%ld\n", (long)sizeof(unsigned char)); printf("sizeof long long=%ld\n", (long)sizeof(long long)); printf("sizeof unsigned long long=%ld\n", (long)sizeof(unsigned long long)); printf("sizeof size_t=%ld\n", (long)sizeof(size_t)); printf("sizeof time_t=%ld\n", (long)sizeof(time_t)); printf("sizeof wchar_t=%ld\n", (long)sizeof(wchar_t)); printf("sizeof mode_t=%ld\n", (long)sizeof(mode_t)); printf("sizeof pid_t=%ld\n", (long)sizeof(pid_t)); sizeof short=2 sizeof unsigned short=2 sizeof int=4 sizeof unsigned int=4 sizeof long=8 sizeof unsigned long=8 sizeof signed char=1 sizeof unsigned char=1 sizeof long long=8 sizeof unsigned long long=8 sizeof size_t=8 sizeof time_t=8 sizeof wchar_t=4 sizeof mode_t=2 sizeof pid_t=4 Traceback (most recent call last): File "translate.py", line 300, in <module> main() File "translate.py", line 205, in main targetspec_dic, translateconfig, config, args = parse_options_and_load_target() File "translate.py", line 153, in parse_options_and_load_target targetspec_dic = load_target(targetspec) File "translate.py", line 105, in load_target mod = __import__(specname) File "targetpypystandalone.py", line 5, in <module> from pypy.objspace.std.objspace import StdObjSpace File "/place/home/hitrobot/pypy-trunk/pypy/objspace/std/__init__.py", line 1, in <module> from pypy.objspace.std.objspace import StdObjSpace File "/place/home/hitrobot/pypy-trunk/pypy/objspace/std/objspace.py", line 3, in <module> from pypy.interpreter import pyframe, function, special File "/place/home/hitrobot/pypy-trunk/pypy/interpreter/pyframe.py", line 13, in <module> from pypy.rlib import jit, rstack File "/place/home/hitrobot/pypy-trunk/pypy/rlib/rstack.py", line 10, in <module> from pypy.rpython.lltypesystem import rffi, lltype File "/place/home/hitrobot/pypy-trunk/pypy/rpython/lltypesystem/rffi.py", line 824, in <module> sys.maxint, sizeof(lltype.Signed))) AssertionError: Mixed configuration of the word size of the machine: the underlying Python was compiled with maxint=2147483647, but the C compiler says that 'long' is 8 bytes I got this error, then I added "-m32" flag to gcc argument list (in the posix.py source file) , but error occured again :( Does anybody tryed to build pypy-c on the 64bit FreeBsd? Thank you for your answer! -- Shadrin Eugene

5 7

Updated 'High Performance Python' tutorial (the one from EuroPython 2011)
by Ian Ozsvald 16 Nov '11

16 Nov '11

Dear all, I've published v0.2 of my High Performance Python tutorial write-up from the session I ran at EuroPython: http://ianozsvald.com/2011/07/25/high-performance-python-tutorial-v0-2-from… Antonio - you asked earlier if the 'expanded math' version of the Mandelbrot solver (using doubles rather than complex numbers) would be faster - I've timed it and it is a bit faster with a nightly build of PyPy, but nowhere near as fast at ShedSkin's generated C output (details below). Maciej - thanks for pointing me at the numpy module. I've added a tiny section showing numpy in PyPy but I haven't converted the Mandelbrot solver to use it (even finishing v0.2 took longer than I'd thought). I'm hoping that some more exposure in the report might bring in more volunteers from outside. Here's a clip from the report in the PyPy section: "By running pypy pure_python.py 1000 1000 on my MacBook it takes 5.9 seconds, running pypy pure_python_2.py 1000 1000 takes 4.9 seconds. (Ian - the only difference with pure_python_2.py is that local dereferences in the tight loop are moved outside the loop, causing fewer dereference operations) As an additional test (not shown in the graphs) I ran pypy shedskin2.py 1000 1000 which runs the expanded math version of the shedskin variant below (this replaces complex numbers with floats and expands abs to avoid the square root). The shedskin2.py result takes 3.2 seconds (which is still much slower than the 0.4s version compiled using shedskin)." The pure_python src is here: https://github.com/ianozsvald/EuroPython2011_HighPerformanceComputing/tree/… shedskin2.py is available here: https://github.com/ianozsvald/EuroPython2011_HighPerformanceComputing/tree/… I haven't tested whether the warm-up periods for PyPy are significant, possibly they account for much of the difference between ShedSkin and PyPy? I want to revisit this but for the next few weeks I have to go back to other projects. I hope the report brings in some new folk for PyPy, Ian. -- Ian Ozsvald (A.I. researcher, screencaster) ian(a)IanOzsvald.com http://IanOzsvald.com http://SocialTiesApp.com/ http://MorConsulting.com/ http://blog.AICookbook.com/ http://TheScreencastingHandbook.com http://FivePoundApp.com/ http://twitter.com/IanOzsvald

5 8

Developer selection for Py3k and Numpy
by Carl Friedrich Bolz 15 Nov '11

15 Nov '11

Hi all, Now that we are getting in some money for our Py3k [1] and Numpy [2] funding proposals (thank you very very much, for everybody who contributed!) it is time to think more concretely about the actual execution. Therefore I want to ask for PyPy developers that are interested in getting paid for their work on the first steps of the Numpy or Py3k proposals to step forward. To be applicable you need to be an experienced PyPy developer who worked in this area before (Numpy) or on the Python interpreter (Py3k). Based on these answers we will then select and announce developers to work on the proposals. The work will be started at the upcoming Gothenburg sprint. Cheers, Carl Friedrich [1] http://pypy.org/numpydonate.html [2] http://pypy.org/py3donate.html

7 9

Success histories needed
by Maciej Fijalkowski 31 Oct '11

31 Oct '11

Hi We're gathering success stories for the website. Anyone feel like providing name, company and some data how pypy worked for them (we accept info how it didn't work on bugs.pypy.org all the time :) Cheers, fijal

4 6

Goteborg sprint - newcomer
by Mark Pearse 30 Oct '11

30 Oct '11

Hi, I would like to come to the next sprint in Goteborg. I have been following the Pypy project with interest since its inception, but this is this first time I have been able to arrange free time to come and participate. I do not have any particular focus for my interest and would be happy to work on whatever people suggest. My degree was in Mathematics, but I have worked most of my life as a programmer. I did an occam compiler for the Manchester dataflow machine as an MSc project. I heard about Pypy while researching for my own (now dead) project to do an opensource Prograph implementation. I will be making my travel arrangements tomorrow. At the moment I plan to leave Brussels by train on the 1st, probably arriving Goteborg early afternoon on the 2nd, and to travel back on the 10th. I shall be looking for cheap accommodation and will try to book the Veckobostader as suggested in the blog. I would be happy to share a room if anyone wants to. Any comments or suggestions about travel or accommodation would be gratefully received. Best regards to all, Mark Pearse

3 2

numpy and multi-dimensional arrays
by matti picus 30 Oct '11

30 Oct '11

So it turns out there are two branches for multi dimensional arrays. I just committed get/set for ndim slices on the branch "numpy NDimArray" , and it turns out I am duplicating work done by someone else on the numpy-multidim branch :( . Could that developer please contact me directly so we can coordinate and me things forward faster without the duplication? BTW - I took the route that left the SingleDimArray intact, if someone would care to write a speed test we could check the comparitive performance between a=numpy.zeros(1000000); a+a #SingleDimArray and a=numpy.zeros((1000000,)); a+a #NDimArray Matti

2 2

PyPy's sandbox used as a class help
by Maciej Fijalkowski 29 Oct '11

29 Oct '11

I thought people might be interested in that: http://pdos.csail.mit.edu/6.858/2011/labs/lab4.html

1 0

Where is `float.is_integer`?
by Ram Rachum 29 Oct '11

29 Oct '11

Guys, am I missing something? ----------------------------- *Python 2.7.2* (default, Jun 12 2011, 15:08:59) [MSC v.1500 32 bit (Intel)] on win32Type "help", "copyright", "credits" or "license" for more information. >>> x = 4.5 >>> x.is_integer() False ----------------------------- Python 2.7.1 (080f42d5c4b4, Aug 23 2011, 11:41:11) [*PyPy 1.6.0* with MSC v.1500 32 bit] on win32 Type "help", "copyright", "credits" or "license" for more information. And now for something completely different: ``- It's hard to say exactly what constitutes research in the computer world, but as a first approximation, it's software that doesn't have users.'' >>>> x = 4.5 >>>> x.is_integer() Traceback (most recent call last): File "<console>", line 1, in <module> AttributeError: 'float' object has no attribute 'is_integer' -----------------------------

4 4