Mailman 3 March 2012 - pypy-dev

HTMLParser, patch
by Alexandre Bm March 11, 2012

March 11, 2012

Hello everyone, i want to submit a patch to HTMLParser.py, is there any one who know where can i submit it ? Thanks ! -- Nobless Oblige ! Alexandre BM

4 4

Excellent work!
by Bent Phillipsen March 10, 2012

March 10, 2012

Thank you for making pypy available. It is really excellent work! I tried pypy on some numerical analysis algorithms and compered to fortran and cpython: On a loop over romberg-integrations I found pypy 2.3 times faster than gfortran without optimization and 2.1 times faster than gfortran with optimization (yes, pypy faster than fortran!). Compared to cpython, pypy is about 25 times faster. The results of the computations with pypy and cpython are identical. On a loop over akima-interpolations pypy is 5 times slower than gfortran, but about 20 times faster than cpython. These results are really impressive. With pypy numerical analysis can be performed efficiently in pure python using lists as arrays (i.e. without using numpy). By the way: I have observed, that the assert statement is not fully implemented in pypy (which possibly is well known). In cpython with the -O option no bytecode is generated for the assert statement. In pypy the same bytecode is generated with and without the -O option. Best regards Bent Phillipsen

3 2

Re: [pypy-dev] How to get PIL to install?
by Matti Picus March 10, 2012

March 10, 2012

I created a fork of Pillow at git://github.com/mattip/Pillow.git for win32 Using a pypy nightly and the github code should allow you to "setup.py install" Pillow for win32. Note you may need some external libraries like libjpeg and zlib to get full functionality. I also fixed a bug in pypy so that "setup.py bdist_wininst" works, maybe when 1.8.1 is released we can think about how to make binary packages publicly available? mattip

1 0

Online Python interpreter on top of PyPy
by kracethekingmaker March 9, 2012

March 9, 2012

Hi PyPyians Thanks to your support for making this happen. Especially to fijal, who was patient enough to answer my question. www.pylive.codespeaks.in, an online python interpreter(strictly pypy interpreter) on top pypy sandboxed version 1.7. Source code of the site is located @ www.github.com/kracekumar/pylive, just feel free to try and post your feedback, in case pypy community wants any analysis or what kind of code ppl try, I can share with the group, please feel free to comment. -- "Talk is cheap, show me the code" -- Linus Torvalds Regards Kracekumar.R www.kracekumar.com

4 6

Py_DecRef() in cpyext
by Stefan Behnel March 8, 2012

March 8, 2012

Hi, having rewritten Cython's dict iteration to use normal iteration, I ran a little benchmark for comparison and it turned out to be (algorithmically) faster by more than an order of magnitude for a dict with 1000 items. That's pretty decent, although it's still about 100x slower than in CPython. I'm sure and certain that there's much more of this low hanging fruit to find all over the place. Next, I ran callgrind over my little benchmark and it showed that the dominating part of the running time had shifted from 99% in PyDict_Next() to 56% in Py_DecRef(). Sadly, there wasn't much more to be read out of it, because the graph was just full of hex numbers instead of names. To be attributed to the JIT compiled code, I guess. Looking at Py_DecRef(), however, left me somewhat baffled. I would have expected this to be the most intensively tuned function in all of cpyext, but it even started with this comment: """ # XXX Optimize these functions and put them into macro definitions """ Ok, sounds reasonable. Should have been done already, though. Especially when I took a look at object.h and saw that the Py_DECREF() macro *always* calls into it. Another surprise. I had understood in previous discussions that the refcount emulation in cpyext only counts C references, which I consider a suitable design. (I guess something as common as Py_None uses the obvious optimisation of always having a ref-count > 1, right? At least when not debugging...) So I changed the macros to use an appropriate C-level implementation: """ #define Py_INCREF(ob) ((((PyObject *)ob)->ob_refcnt > 0) ? \ ((PyObject *)ob)->ob_refcnt++ : (Py_IncRef((PyObject *)ob))) #define Py_DECREF(ob) ((((PyObject *)ob)->ob_refcnt > 1) ? \ ((PyObject *)ob)->ob_refcnt-- : (Py_DecRef((PyObject *)ob))) #define Py_XINCREF(op) do { if ((op) == NULL) ; else Py_INCREF(op); \ } while (0) #define Py_XDECREF(op) do { if ((op) == NULL) ; else Py_DECREF(op); \ } while (0) """ to tell the C compiler that it doesn't actually need to call into PyPy in most cases (note that I didn't use any branch prediction macros, but that shouldn't change all that much anyway). This shaved off a couple of cycles from my iteration benchmark, but much less than I would have liked. My intuition tells me that this is because almost all objects that appear in the benchmark are actually short-lived in C space so that pretty much every Py_DECREF() on them kills them straight away and thus calls into Py_DecRef() anyway. To be verified with a better test. I think this could be fixed in some way by figuring out if there is at least one live reference to it inside of PyPy (which is usually the case for container iteration and parameters, for example), and as long as that's the case, add one to the refcount to keep C code from needing to call back into PyPy unnecessarily. A finaliser could do that, I guess. Ok, next, I looked at the Py_DecRef() implementation itself and found it to be rather long. Since I wasn't sure what the JIT would make of it, please don't shout at me if any of the following comments ("^^") make no sense. @cpython_api([PyObject], lltype.Void) def Py_DecRef(space, obj): if not obj: return ^^ this would best be handled right in C space to avoid unnecessary call overhead and to allow the C compiler to drop this test when possible. assert lltype.typeOf(obj) == PyObject obj.c_ob_refcnt -= 1 if DEBUG_REFCOUNT: debug_refcount("DECREF", obj, obj.c_ob_refcnt, frame_stackdepth=3) ^^ ok, I assume debug code will automatically be removed completely if obj.c_ob_refcnt == 0: state = space.fromcache(RefcountState) ptr = rffi.cast(ADDR, obj) if ptr not in state.py_objects_r2w: # this is a half-allocated object, lets call the deallocator # without modifying the r2w/w2r dicts _Py_Dealloc(space, obj) ^^ this looks like an emergency special case to me - shouldn't be on the fast path... else: w_obj = state.py_objects_r2w[ptr] ^^ here, I wonder if the JIT is smart enough to detect the double weakref dict lookup in the (I guess) 99% code path and folds it into one? ^^ I actually wonder why this needs to use a weakref dict instead of a normal dict. It's ref-counted after all, so we *know* there's a live reference to it until further notice. Are weakref dicts as fast as normal dicts in PyPy? del state.py_objects_r2w[ptr] w_type = space.type(w_obj) if not w_type.is_cpytype(): _Py_Dealloc(space, obj) del state.py_objects_w2r[w_obj] # if the object was a container for borrowed references state.delete_borrower(w_obj) ^^ in some place here, I would have expected the use of a freelist for the C space object representations in order to avoid unnecessary allocations, especially for things like short tuples etc. Is this hidden somewhere else? Or does PyPy's runtime handle this kind of caching internally somewhere? else: if not we_are_translated() and obj.c_ob_refcnt < 0: message = "Negative refcount for obj %s with type %s" % ( obj, rffi.charp2str(obj.c_ob_type.c_tp_name)) print >>sys.stderr, message assert False, message ^^ This will also vanish automatically, right? So, from a quick and shallow look, it seems to me that this code could be improved. By taking work off the fast paths and avoiding create-delete cycles, something as ubiquitous as this function could help in making C-API code run a lot faster. Stefan

2 5

Seeking a bit of guidance
by Russel Winder March 7, 2012

March 7, 2012

For some reason I had assumed that pypy could be a drop in replacement for python. I am on Debian Unstable and I am suspected that my assumption is being thwarted. In particular in trying to run SCons. |> python /home/Checkouts/Mercurial/SCons/bootstrap/src/script/scons.py scons: Reading SConscript files ... scons: done reading SConscript files. scons: Building targets ... stopServerIfNeedBe(["allPDFs"], []) scons: done building targets. |> pypy /home/Checkouts/Mercurial/SCons/bootstrap/src/script/scons.py Traceback (most recent call last): File "app_main.py", line 51, in run_toplevel File "/home/Checkouts/Mercurial/SCons/bootstrap/src/script/scons.py", line 187, in <module> import SCons.Script ImportError: No module named SCons This may be a SCons problem or a Debian Python packaging issue but higher likelihood is a PyPy configuration issue? Basically I am not sure where to start. I use SCons from a Mercurial clone, Python from Debian packages, PyPy from then downloaded binary distribution. |> python --version Python 2.7.2+ |> pypy --version Python 2.7.1 (7773f8fc4223, Nov 18 2011, 18:47:11) [PyPy 1.7.0 with GCC 4.4.3] Help and guidance greatfully received. Thanks. -- Russel. ============================================================================= Dr Russel Winder t: +44 20 7585 2200 voip: sip:russel.winder@ekiga.net 41 Buckmaster Road m: +44 7770 465 077 xmpp: russel(a)russel.org.uk London SW11 1EN, UK w: www.russel.org.uk skype: russel_winder

4 8

pypy2exe
by Matti Picus March 6, 2012

March 6, 2012

3 2

web_review
by Affleck, Melanie March 2, 2012

March 2, 2012

Dear Pypy-dev, We will review your site and your web placement, at no-cost to you, now. Reply today for your complimentary analysis. Sincerely, Melanie Affleck Interplay_Marketing pypy-dev(a)codespeak.net 2/29/2012

4 4