Mailman 3 April 2012 - pypy-dev

Python vs pypy: interesting performance difference [dict.setdefault]
by David Naylor 10 Mar '14

10 Mar '14

Hi, I needed to create a cache of date and time objects and I wondered what was the best way to handle the cache. For comparison I put together the following test: <file="iforkey.py"> import datetime import random import timeit ranges = [datetime.datetime(2011,01, random.randint(1, 31)) for i in xrange(1000)] def ifdict(): cache = {} b = [] for i in ranges: key = i.day if key in cache: b.append(cache[key]) else: date = i.date() cache[key] = date b.append(date) def keydict(): cache = {} b = [] for i in ranges: key = i.day try: b.append(cache[key]) except KeyError: date = i.date() cache[key] = date b.append(date) def defaultdict(): cache = {} b= [] for i in ranges: b.append(cache.setdefault(i, i.date())) print "ifdict:", timeit.repeat("ifdict()", "from __main__ import ifdict", number=10000) print "keydict:", timeit.repeat("keydict()", "from __main__ import keydict", number=10000) print "defaultdict:", timeit.repeat("defaultdict()", "from __main__ import defaultdict", number=10000) </file> # python iforfile.py ifdict: [2.432887077331543, 2.4002890586853027, 2.397233009338379] keydict: [2.3483030796051025, 2.358638048171997, 2.314802885055542] defaultdict: [3.5384328365325928, 3.5329859256744385, 3.5728111267089844] # pypy iforfile.py (pypy 1.5) ifdict: [0.8129069805145264, 0.74648118019104, 0.7432689666748047] keydict: [0.5187451839447021, 0.4662129878997803, 0.4504108428955078] defaultdict: [37.98510789871216, 37.859113931655884, 37.92770600318909] Pypy displays significant slowdown in the defaultdict function, otherwise displays its usual speedup. To check what is the cause I replaced i.date() with i.day and found no major difference in times. It appears dict.setdefault (or it's interaction with jit) is causing a slow down. Regards

4 14

Patches for Pypy under cygwin
by Uwe F. Mayer 15 Jun '12

15 Jun '12

I edited / hacked the sources and managed to compile PyPy under Cygwin based on Python 2.6.7 and gcc-4.5.3. If you find this of interest, the patches are available at http://www.tux.org/~mayer/cygwin/pypy/. In particular the work-around for the tm structure to get by without the tm_gmtoff and tm_zone fields may be of interest, as it allows compilation of the rctime module on systems that don't have it, such as some versions of SunOS. --Uwe

5 20

update (+patch) on embedding pypy
by Roberto De Ioris 09 Jun '12

09 Jun '12

Hi everyone, i have finally managed to have a pypy plugin into uWSGI via libpypy-c. The first step was exporting a bunch of symbols in libpypy-c: "RPython_StartupCode", "pypy_g_ObjSpace_startup", "pypy_g_call_startup", "pypy_g_State_startup", "pypy_g_Module_startup", "pypy_g_pypy_module_cpyext_state_State" pypy_g_ObjSpace_startup and pypy_g_Module_startup are not strictly required, i have used them only for debugging some very specific areas. Attached there is a patch for latest tip. After that, i have defined a Py_Initialize() replacement: http://projects.unbit.it/uwsgi/browser/plugins/python/python_plugin.c#L8 and placed a bunch of #ifdef for unsupported functions (like PyFile_FromFile) Finally i have built libpypy with pypy translate.py -Ojit --shared and the uWSGI plugin with LDFLAGS="-L." pypy-c uwsgiconfig.py --plugin plugins/pypy core (LDFLAGS is used as i have libpypy-c in the current dir) Run uWSGI with the welcome app (and a bunch of processes): uwsgi --http-socket :8080 --plugin pypy --module welcome --master --processes 8 Incredibly the whole uwsgi api (that it is pretty complex, involving tons of c code) works flawlessly as well as the custom wsgi.input object (the whole upload test suite passes). Running python threads from the WSGI app works too, but mapping python threads to already available pthreads is a no go (all explodes, i will investigate on that) Importing applications as modules works normally, but importing via file requires implementing a bunch of c-api in pypy (i will address that soon) virtualenv, sys.argv and setting programname is not supported (no c api available) i can get rid of this too in the next few days. Important thing: i have tested it only on OSX -- Roberto De Ioris http://unbit.it

6 23

[Repost] PyPy 1.8 50% slower than PyPy 1.7 on my test case
by Laurent Vaucher 04 Jun '12

04 Jun '12

Sorry to repost, but does anyone have an idea about what I could do to track down the source of the slowdown? Should I run with some trace to try to compare how different parts behave? How could I do that? Thanks, Laurent. Hi and first of all, thanks for that great project. Now to my "problem". I'm doing some puzzle-solving, constraint processing with Python and on my particular program, PyPy 1.8 showed a 50% increase in running time over PyPy 1.7. I'm not doing anything fancy (no numpy, itertools, etc.). The program is here: https://github.com/slowfrog/hexiom To reproduce: - fetch hexiom2.py and level36.txt from github - run 'pypy hexiom2.py -sfirst level36.txt' On my machine (Windows 7) the timings are the following: Python 2.6.5/Windows 32 3m35s Python 2.7.1 (930f0bc4125a, Nov 27 2011, 11:58:57) [PyPy 1.7.0 with MSC v.1500 32 bit] 31s Python 2.7.2 (0e28b379d8b3, Feb 09 2012, 18:31:47) [PyPy 1.8.0 with MSC v.1500 32 bit] 48s I'm using the default options. Do you have any idea what could be causing that? Thanks, Laurent.

3 5

Question about stm_descriptor_init(), tasklets and OS threads
by Andrew Francis 10 May '12

10 May '12

Hi Armin: I am looking at stm_descriptor_init(). Right now makes a call to pthread_self(). In a potential Stackless prototype, I would want it to get the current tasklet instead. Shouldn't this be enough to get a trivial implementation of Stackless (by trivial, one thread. Hopefully by sticking to a single thread we don't have to alter any low level locking stuff) to interact with the transaction module. Enough to analyse programmes? Cheers, Andrew

2 6

Biting off more than I can chew
by Greg Bowyer 07 May '12

07 May '12

The other night I got this burning desire to recreate the Azul GPGC (the java pauseless collector) inside pypy, with a view that it could be used alongside the STM work to make pypy a low (no?) pause concurrent VM. When I started tackling the code I realised I might have bitten off a little more than I can chew. My question (probably one of many to irritate and annoy all the fine folks here) would be, is there a sensible way to compile into pypy a small amount of C code that can be used to bootstrap and bridge some esoteric c libraries into pypy, the code that I want to run, on startup of pypy would be the following https://bitbucket.org/GregBowyer/pypy-c4gc/changeset/0de575b3a8d1#chg-azm_m… I have noticed that there are a few bridges into direct c code in the pypy source, but I cannot fathom how these are linked into RPython -- Greg

3 5

custom classes in applevel tests
by Timo Paulssen 30 Apr '12

30 Apr '12

Hello there, I was writing a few test cases for numpypy to make sure it behaves like numpy does when confronted with objects that have either an __int__, an __index__ or both (rule of thumb: index is prefered, but int is accepted in absence of index). Now, I wrote six tests that use three custom objects (one set for getitem, one set for setitem and one test each for int only, int and index and index only). I couldn't put those classes outside the applevel test (this is about pypy/module/micronumpy/test/test_numarray.py btw) because there's some kind of separation going on there. I *thought* the way to do it was to put it inside the applevel test class and have its name start with w_, but that wasn't it either. How do I make this work? Currently I have copies of the classes in each of the tests, but I'd like to clean this up. After this test cleanup, I think the branch numpypy-issue1137 is ready to be merged. Buildbot seems to agree. - Timo

3 2

custom classes in applevel tests
by Timo Paulssen 29 Apr '12

29 Apr '12

Hello there, I was writing a few test cases for numpypy to make sure it behaves like numpy does when confronted with objects that have either an __int__, an __index__ or both (rule of thumb: index is prefered, but int is accepted in absence of index). Now, I wrote six tests that use three custom objects (one set for getitem, one set for setitem and one test each for int only, int and index and index only). I couldn't put those classes outside the applevel test (this is about pypy/module/micronumpy/test/test_numarray.py btw) because there's some kind of separation going on there. I *thought* the way to do it was to put it inside the applevel test class and have its name start with w_, but that wasn't it either. How do I make this work? Currently I have copies of the classes in each of the tests, but I'd like to clean this up. After this test cleanup, I think the branch numpypy-issue1137 is ready to be merged. Buildbot seems to agree. - Timo

1 0

windows and buildbots
by Matti Picus 27 Apr '12

27 Apr '12

2 1

Placement/ Analysis/ Web
by Josh Soares 27 Apr '12

27 Apr '12

pypy-dev(a)codespeak.net : Our company can put your web site in the best positions on the major search engines. We will give you a complimentary site analysis and show you how you can change your placement immediately. We have done this for thousands of companies. Ask us to prove it. And this won't cost a fortune. Reply to us for your web site quote. Don't forget to include how we can reach you. Sincerely, Josh Soares

1 0

pypy-dev April 2012