Mailman 3 May 2013 - pypy-dev

performance issue with context managers
by Jasper Spaans May 16, 2013

May 16, 2013

Hi list, I was toying around a bit with writing a statistical profiler in python, and came up with https://gist.github.com/jap/5584946 For reference, the main routine is: with Profiler() as p: with ProfilerContext("c1"): s = "" for t in range(100000): with ProfilerContext("c2"): s = s + "a" s = s + "b" print p.get_data() When running it on my local machine, this gives the following output: spaans@spaans-e6500:/tmp$ /usr/… [View More]bin/time ~/xsrc/pypy-2.0/bin/pypy pmp.py Counter({'c1': 638}) 6.42user 0.42system 0:07.06elapsed 97%CPU (0avgtext+0avgdata 30160maxresident)k 0inputs+0outputs (0major+8383minor)pagefaults 0swaps spaans@spaans-e6500:/tmp$ /usr/bin/time python pmp.py Counter({'c1': 18, 'c2': 3}) 0.23user 0.02system 0:00.25elapsed 98%CPU (0avgtext+0avgdata 8200maxresident)k 0inputs+0outputs (0major+2226minor)pagefaults 0swaps So, two things seem to happen: the pypy version is almost 30 times slower than the python version (but hey, string appending has poor performance), and it somehow does not trigger the "c2" context.. Is the c2 context supposed to disappear? If I change the main loop to for t in range(100000): with ProfilerContext("c2"): s = s + "a" The output is still limited to spaans@spaans-e6500:/tmp$ /usr/bin/time ~/xsrc/pypy-2.0/bin/pypy pmp.py Counter({'c1': 156}) 1.61user 0.14system 0:01.78elapsed 98%CPU (0avgtext+0avgdata 30040maxresident)k which seems credible, because when I change it to for t in range(100000): s = s + "a" it's suddenly fast in pypy as well: 0.03user 0.01system 0:00.05elapsed 96%CPU (0avgtext+0avgdata 8024maxresident)k Removing all the threading.local stuff gives me the following performance data: spaans@spaans-e6500:/tmp$ /usr/bin/time python pmp.py Counter({'c1': 12, 'c2': 6}) 0.20user 0.01system 0:00.22elapsed 98%CPU (0avgtext+0avgdata 8192maxresident)k 0inputs+0outputs (0major+2224minor)pagefaults 0swaps spaans@spaans-e6500:/tmp$ /usr/bin/time ~/xsrc/pypy-2.0/bin/pypy pmp.py Counter({'c1': 621}) 6.18user 0.42system 0:06.76elapsed 97%CPU (0avgtext+0avgdata 30084maxresident)k which does not seem to differ that much. Finally, to rule out object creation issues, main was changed to: with Profiler() as p: with ProfilerContext("c1"): p2 = ProfilerContext("c2") s = "" for t in range(100000): with p2: s = s + "a" s = s + "b" print p.get_data() but that still behaves similar to the previous runs: spaans@spaans-e6500:/tmp$ /usr/bin/time ~/xsrc/pypy-2.0/bin/pypy pmp.py Counter({'c1': 624}) 6.25user 0.38system 0:06.69elapsed 99%CPU (0avgtext+0avgdata 29792maxresident)k spaans@spaans-e6500:/tmp$ /usr/bin/time python pmp.py Counter({'c2': 6, 'c1': 6}) 0.14user 0.02system 0:00.16elapsed 98%CPU (0avgtext+0avgdata 8188maxresident)k (all of this on my trusty old Dell E6500 running Ubuntu 13.04 on amd64) Could someone help me get "c2" registered, or is this expected behaviour? :) Cheers, Jasper -- /\____/\ ir. Jasper Spaans // Lead Developer DetACT \ (_)/ Fox-IT - For a more secure society! \ X T: +31-15-2847999 \ / \ M: +31-6-41588725 \/ KvK Haaglanden 27301624 [View Less]

3 2

Include .so modules generated with f2py from fortran script
by Fabio D'Orta May 15, 2013

May 15, 2013

Hello to everyone, I'm new to python and especially to pypy. I have developed a code in Cpython with some modules, that were the bottle-neck, in fortran thanks to f2py. I'm translating the code as pypy could run (changing def that uses numpy not supported by numpypy) but I'm stuck in finding a way to ingest the .so module into pypy. I have tried with CFFI but with no luck. I have also considered to write the fortran modules (which basically solves linear equation and find the eigenvalues) in … [View More]

2 4

SSL version?
by Dan Stromberg May 14, 2013

May 14, 2013

I get an error when trying to run pypy 2.0 on a Debian Wheezy system: dstromberg@deskie:~/src/home-svn/backshift/trunk$ /usr/local/pypy-2.0/bin/pypy /usr/local/pypy-2.0/bin/pypy: error while loading shared libraries: libssl.so.0.9.8: cannot open shared object file: No such file or directory dstromberg@deskie:~/src/home-svn/backshift/trunk$ Isn't libssl 0.9.8 getting kind of old? I don't see it in synaptic.

6 5

Salam Sahajetra
by Widiyana Samudra May 13, 2013

May 13, 2013

Assalamualaikum Wr.Wb... Sebelum dan sesudahnya saya ingin memperkenalkan diri saya,nama saya WIDIYANA SAMUDRA berasal dari Indonesia dan bekerja di London (United Kingdom), ingin berkenalan dengan anda di sana. Saya ingin menawarkan satu peluang bisnis yang begitu bagus,dan bagi Anda yang berminat dengan bisnis ... Inilah kesempatan Anda,kapan lagi kalau bukan sekarang bukan.??? Produk ini di namakan (MULITE CLEANSER) kegunaannya untuk mencuci barangan yang sangat berharga sekali seperti … [View More]

1 0

Issue 1475 (add_memory_pressure())
by Bart Wiegmans May 12, 2013

May 12, 2013

Hi everybody, First, let me introduce myself. I'm Bart Wiegmans and I would like to work for pypy via GSoC this summer. To that end I proposed adding a concurrent / incremental garbage collector to PyPy. Reasonably, Fijal requested me to bring in some code before I could be accepted, and since issue 1475 (https://bugs.pypy.org/issue1475) is rather memory-related, it seems like a good fit. However, I've failed to make any progress on it (despite advice). In fact, I'd say that at this point I … [View More]

1 0

Using "rpath"
by Armin Rigo May 12, 2013

May 12, 2013

Hi all, I think I know how to solve the two linking issues we were discussing on irc yesterday: using the "rpath" (thanks squeaky_pl for originally pointing it out). See for example http://stackoverflow.com/a/6323222/1556290 . 1) This could let us distribute more self-contained executables by putting the libffi.so, libssl.so, etc. along it in the same directory, and specifying an rpath of "$ORIGIN". 2) This could be used for building in "--shared" mode, with a small executable and a big … [View More]

3 5

cffi wrapper for libeinspline, a cubic B-spline library, runs on Pypy
by Kay F. Jahnke May 12, 2013

May 12, 2013

Hi group! I've been working on wrapping libeinspline, a comprehensive C library for creating and evaluating cubic B-splines in 1-3 dimensions and real and complex data types. The wrap is done using cffi, and it runs with Pypy and Cpython. I've now put an alpha version online at https://bitbucket.org/kfj/python-bspline I am pointing you to this project since it actually works well with Pypy and has a few bits in it which you may find inspiring. I'd welcome comments. I wrote the wrapper … [View More]

1 0

PyPy 2.0 - Einstein Sandwich
by Maciej Fijalkowski May 11, 2013

May 11, 2013

We're pleased to announce PyPy 2.0. This is a stable release that brings a swath of bugfixes, small performance improvements and compatibility fixes. PyPy 2.0 is a big step for us and we hope in the future we'll be able to provide stable releases more often. You can download the PyPy 2.0 release here: http://pypy.org/download.html The two biggest changes since PyPy 1.9 are: * stackless is now supported including greenlets, which means eventlet and gevent should work (but read below … [View More]about gevent) * PyPy now contains release 0.6 of `cffi`_ as a builtin module, which is preferred way of calling C from Python that works well on PyPy .. _`cffi`: http://cffi.readthedocs.org If you're using PyPy for anything, it would help us immensely if you fill out the following survey: http://bit.ly/pypysurvey This is for the developers eyes and we will not make any information public without your agreement. What is PyPy? ============= PyPy is a very compliant Python interpreter, almost a drop-in replacement for CPython 2.7. It's fast (`pypy 2.0 and cpython 2.7.3`_ performance comparison) due to its integrated tracing JIT compiler. This release supports x86 machines running Linux 32/64, Mac OS X 64 or Windows 32. Windows 64 work is still stalling, we would welcome a volunteer to handle that. ARM support is on the way, as you can see from the recently released alpha for ARM. .. _`pypy 2.0 and cpython 2.7.3`: http://speed.pypy.org Highlights ========== * Stackless including greenlets should work. For gevent, you need to check out `pypycore`_ and use the `pypy-hacks`_ branch of gevent. * cffi is now a module included with PyPy. (`cffi`_ also exists for CPython; the two versions should be fully compatible.) It is the preferred way of calling C from Python that works on PyPy. * Callbacks from C are now JITted, which means XML parsing is much faster. * A lot of speed improvements in various language corners, most of them small, but speeding up some particular corners a lot. * The JIT was refactored to emit machine code which manipulates a "frame" that lives on the heap rather than on the stack. This is what makes Stackless work, and it could bring another future speed-up (not done yet). * A lot of stability issues fixed. * Refactoring much of the numpypy array classes, which resulted in removal of lazy expression evaluation. On the other hand, we now have more complete dtype support and support more array attributes. .. _`pypycore`: https://github.com/gevent-on-pypy/pypycore/ .. _`pypy-hacks`: https://github.com/schmir/gevent/tree/pypy-hacks Cheers, fijal, arigo and the PyPy team [View Less]

7 12

Re: [pypy-dev] [pypy-commit] cffi default: Try to preserve the exact error message
by Philip Jenvey May 10, 2013

May 10, 2013

Could you please switch this to Py3 compat syntax, KeyError as e? On May 10, 2013, at 7:07 AM, arigo wrote: > Author: Armin Rigo <arigo(a)tunes.org> > Branch: > Changeset: r1250:452b57d57304 > Date: 2013-05-10 16:07 +0200 > http://bitbucket.org/cffi/cffi/changeset/452b57d57304/ > > Log: Try to preserve the exact error message > > diff --git a/cffi/api.py b/cffi/api.py > --- a/cffi/api.py > +++ b/cffi/api.py > @@ -372,8 +372,8 @@ > … [View More]

1 0

x is y <=> id(x)==id(y)
by Armin Rigo May 10, 2013

May 10, 2013

Hi all, I'm just wondering again about some "bug" reports that are not bugs, about people misusing "is" to compare two immutable objects. The current situation in PyPy is that "is" works like "==" for ints, longs, floats or complexes. It does not for strs or unicodes or tuples. Now of course someone on python-dev was (indirectly) complaining that you can compare in CPython ``x is ' '``, which works because single-character strings are cached, but not in PyPy. I'm sure someone else has been … [View More]

11 20