Mailman 3 May 2013 - pypy-dev

performance issue with context managers
by Jasper Spaans May 16, 2013

May 16, 2013

Hi list, I was toying around a bit with writing a statistical profiler in python, and came up with https://gist.github.com/jap/5584946 For reference, the main routine is: with Profiler() as p: with ProfilerContext("c1"): s = "" for t in range(100000): with ProfilerContext("c2"): s = s + "a" s = s + "b" print p.get_data() When running it on my local machine, this gives the following output: spaans@spaans-e6500:/tmp$ /usr/bin/time ~/xsrc/pypy-2.0/bin/pypy pmp.py Counter({'c1': 638}) 6.42user 0.42system 0:07.06elapsed 97%CPU (0avgtext+0avgdata 30160maxresident)k 0inputs+0outputs (0major+8383minor)pagefaults 0swaps spaans@spaans-e6500:/tmp$ /usr/bin/time python pmp.py Counter({'c1': 18, 'c2': 3}) 0.23user 0.02system 0:00.25elapsed 98%CPU (0avgtext+0avgdata 8200maxresident)k 0inputs+0outputs (0major+2226minor)pagefaults 0swaps So, two things seem to happen: the pypy version is almost 30 times slower than the python version (but hey, string appending has poor performance), and it somehow does not trigger the "c2" context.. Is the c2 context supposed to disappear? If I change the main loop to for t in range(100000): with ProfilerContext("c2"): s = s + "a" The output is still limited to spaans@spaans-e6500:/tmp$ /usr/bin/time ~/xsrc/pypy-2.0/bin/pypy pmp.py Counter({'c1': 156}) 1.61user 0.14system 0:01.78elapsed 98%CPU (0avgtext+0avgdata 30040maxresident)k which seems credible, because when I change it to for t in range(100000): s = s + "a" it's suddenly fast in pypy as well: 0.03user 0.01system 0:00.05elapsed 96%CPU (0avgtext+0avgdata 8024maxresident)k Removing all the threading.local stuff gives me the following performance data: spaans@spaans-e6500:/tmp$ /usr/bin/time python pmp.py Counter({'c1': 12, 'c2': 6}) 0.20user 0.01system 0:00.22elapsed 98%CPU (0avgtext+0avgdata 8192maxresident)k 0inputs+0outputs (0major+2224minor)pagefaults 0swaps spaans@spaans-e6500:/tmp$ /usr/bin/time ~/xsrc/pypy-2.0/bin/pypy pmp.py Counter({'c1': 621}) 6.18user 0.42system 0:06.76elapsed 97%CPU (0avgtext+0avgdata 30084maxresident)k which does not seem to differ that much. Finally, to rule out object creation issues, main was changed to: with Profiler() as p: with ProfilerContext("c1"): p2 = ProfilerContext("c2") s = "" for t in range(100000): with p2: s = s + "a" s = s + "b" print p.get_data() but that still behaves similar to the previous runs: spaans@spaans-e6500:/tmp$ /usr/bin/time ~/xsrc/pypy-2.0/bin/pypy pmp.py Counter({'c1': 624}) 6.25user 0.38system 0:06.69elapsed 99%CPU (0avgtext+0avgdata 29792maxresident)k spaans@spaans-e6500:/tmp$ /usr/bin/time python pmp.py Counter({'c2': 6, 'c1': 6}) 0.14user 0.02system 0:00.16elapsed 98%CPU (0avgtext+0avgdata 8188maxresident)k (all of this on my trusty old Dell E6500 running Ubuntu 13.04 on amd64) Could someone help me get "c2" registered, or is this expected behaviour? :) Cheers, Jasper -- /\____/\ ir. Jasper Spaans // Lead Developer DetACT \ (_)/ Fox-IT - For a more secure society! \ X T: +31-15-2847999 \ / \ M: +31-6-41588725 \/ KvK Haaglanden 27301624

3 2

Include .so modules generated with f2py from fortran script
by Fabio D'Orta May 15, 2013

May 15, 2013

Hello to everyone, I'm new to python and especially to pypy. I have developed a code in Cpython with some modules, that were the bottle-neck, in fortran thanks to f2py. I'm translating the code as pypy could run (changing def that uses numpy not supported by numpypy) but I'm stuck in finding a way to ingest the .so module into pypy. I have tried with CFFI but with no luck. I have also considered to write the fortran modules (which basically solves linear equation and find the eigenvalues) in pypy but linalg is missing in numpypy. Could you help me? Thanks in advance for your time Fabio

2 4

SSL version?
by Dan Stromberg May 14, 2013

May 14, 2013

I get an error when trying to run pypy 2.0 on a Debian Wheezy system: dstromberg@deskie:~/src/home-svn/backshift/trunk$ /usr/local/pypy-2.0/bin/pypy /usr/local/pypy-2.0/bin/pypy: error while loading shared libraries: libssl.so.0.9.8: cannot open shared object file: No such file or directory dstromberg@deskie:~/src/home-svn/backshift/trunk$ Isn't libssl 0.9.8 getting kind of old? I don't see it in synaptic.

6 5

Salam Sahajetra
by Widiyana Samudra May 13, 2013

May 13, 2013

Assalamualaikum Wr.Wb... Sebelum dan sesudahnya saya ingin memperkenalkan diri saya,nama saya WIDIYANA SAMUDRA berasal dari Indonesia dan bekerja di London (United Kingdom), ingin berkenalan dengan anda di sana. Saya ingin menawarkan satu peluang bisnis yang begitu bagus,dan bagi Anda yang berminat dengan bisnis ... Inilah kesempatan Anda,kapan lagi kalau bukan sekarang bukan.??? Produk ini di namakan (MULITE CLEANSER) kegunaannya untuk mencuci barangan yang sangat berharga sekali seperti batu intan permata yg masih mentah. Dan pada waktu yang sama perusahaan di tempat saya bekerja membutuhkan MULITE CLEANSER dimana mineral tersbut tersedia dari operator yg di Indonesia. Jadi bagi anda yang berminat dengan bisnis ini, saya ingin Anda menjadi Agent untuk menjual produk tersebut ke pada perusahaan tempat saya bekerja, (Anda membeli produk tersebut dari operator yang di Indonesia terlebih dahulu dengan harga 500USD(Lima Juta Rupiah) per karton dan dijualnya kembali ke pada perusahaan di tempat saya bekerja dengan harga 1,500USD(Lima belas Juta per karton) Yang paling penting Niat dan ikhlas untuk mencari rezeki semata-mata karena Allah. Insya Allah, Allah akan membantu kita untuk mencari rezeki yang halal dan berkat,saya ingin Anda buat keputusan yang bijak, Anda bisa merubahnya .. Insya Allah ' Apa bila ada berminat atau ada pertanyaan, silahkan balas email terus ke saya berdasarkan alamat di bawah atau silahkan berikan number supaya bisa di hubungi dan saya akan menerangkannya lebih lanjut di telepon: Insya Allah saya mencoba membantu Anda. Semoga ikatan silaturahmi sesama kita diberkati Allah. Selamat berkenalan. widiyana26(a)yahoo.com Yang benar... WIDIYANA SAMUDRA

1 0

Issue 1475 (add_memory_pressure())
by Bart Wiegmans May 12, 2013

May 12, 2013

Hi everybody, First, let me introduce myself. I'm Bart Wiegmans and I would like to work for pypy via GSoC this summer. To that end I proposed adding a concurrent / incremental garbage collector to PyPy. Reasonably, Fijal requested me to bring in some code before I could be accepted, and since issue 1475 (https://bugs.pypy.org/issue1475) is rather memory-related, it seems like a good fit. However, I've failed to make any progress on it (despite advice). In fact, I'd say that at this point I simply don't understand PyPy well enough to add this feature, and I'm not sure what to do next. Anyway, this way you know that I've tried, at least. If anyone has any more advice I'll happily try again, of course. Thank you for your time. Kind regards, Bart Wiegmans

1 0

Using "rpath"
by Armin Rigo May 12, 2013

May 12, 2013

Hi all, I think I know how to solve the two linking issues we were discussing on irc yesterday: using the "rpath" (thanks squeaky_pl for originally pointing it out). See for example http://stackoverflow.com/a/6323222/1556290 . 1) This could let us distribute more self-contained executables by putting the libffi.so, libssl.so, etc. along it in the same directory, and specifying an rpath of "$ORIGIN". 2) This could be used for building in "--shared" mode, with a small executable and a big libpypy.so in the same directory, again using the rpath "$ORIGIN". A bientôt, Armin.

3 5

cffi wrapper for libeinspline, a cubic B-spline library, runs on Pypy
by Kay F. Jahnke May 12, 2013

May 12, 2013

Hi group! I've been working on wrapping libeinspline, a comprehensive C library for creating and evaluating cubic B-splines in 1-3 dimensions and real and complex data types. The wrap is done using cffi, and it runs with Pypy and Cpython. I've now put an alpha version online at https://bitbucket.org/kfj/python-bspline I am pointing you to this project since it actually works well with Pypy and has a few bits in it which you may find inspiring. I'd welcome comments. I wrote the wrapper because I wanted B-splines for a Pypy pet project of mine and couldn't find anything suitable. It may be of use while a Pypy version of scipy hasn't materialized, and even has a few bits which I haven't found in scipy. Kay F. Jahnke

1 0

PyPy 2.0 - Einstein Sandwich
by Maciej Fijalkowski May 10, 2013

May 10, 2013

We're pleased to announce PyPy 2.0. This is a stable release that brings a swath of bugfixes, small performance improvements and compatibility fixes. PyPy 2.0 is a big step for us and we hope in the future we'll be able to provide stable releases more often. You can download the PyPy 2.0 release here: http://pypy.org/download.html The two biggest changes since PyPy 1.9 are: * stackless is now supported including greenlets, which means eventlet and gevent should work (but read below about gevent) * PyPy now contains release 0.6 of `cffi`_ as a builtin module, which is preferred way of calling C from Python that works well on PyPy .. _`cffi`: http://cffi.readthedocs.org If you're using PyPy for anything, it would help us immensely if you fill out the following survey: http://bit.ly/pypysurvey This is for the developers eyes and we will not make any information public without your agreement. What is PyPy? ============= PyPy is a very compliant Python interpreter, almost a drop-in replacement for CPython 2.7. It's fast (`pypy 2.0 and cpython 2.7.3`_ performance comparison) due to its integrated tracing JIT compiler. This release supports x86 machines running Linux 32/64, Mac OS X 64 or Windows 32. Windows 64 work is still stalling, we would welcome a volunteer to handle that. ARM support is on the way, as you can see from the recently released alpha for ARM. .. _`pypy 2.0 and cpython 2.7.3`: http://speed.pypy.org Highlights ========== * Stackless including greenlets should work. For gevent, you need to check out `pypycore`_ and use the `pypy-hacks`_ branch of gevent. * cffi is now a module included with PyPy. (`cffi`_ also exists for CPython; the two versions should be fully compatible.) It is the preferred way of calling C from Python that works on PyPy. * Callbacks from C are now JITted, which means XML parsing is much faster. * A lot of speed improvements in various language corners, most of them small, but speeding up some particular corners a lot. * The JIT was refactored to emit machine code which manipulates a "frame" that lives on the heap rather than on the stack. This is what makes Stackless work, and it could bring another future speed-up (not done yet). * A lot of stability issues fixed. * Refactoring much of the numpypy array classes, which resulted in removal of lazy expression evaluation. On the other hand, we now have more complete dtype support and support more array attributes. .. _`pypycore`: https://github.com/gevent-on-pypy/pypycore/ .. _`pypy-hacks`: https://github.com/schmir/gevent/tree/pypy-hacks Cheers, fijal, arigo and the PyPy team

7 12

Re: [pypy-dev] [pypy-commit] cffi default: Try to preserve the exact error message
by Philip Jenvey May 10, 2013

May 10, 2013

Could you please switch this to Py3 compat syntax, KeyError as e? On May 10, 2013, at 7:07 AM, arigo wrote: > Author: Armin Rigo <arigo(a)tunes.org> > Branch: > Changeset: r1250:452b57d57304 > Date: 2013-05-10 16:07 +0200 > http://bitbucket.org/cffi/cffi/changeset/452b57d57304/ > > Log: Try to preserve the exact error message > > diff --git a/cffi/api.py b/cffi/api.py > --- a/cffi/api.py > +++ b/cffi/api.py > @@ -372,8 +372,8 @@ > BType = ffi._get_cached_btype(tp) > try: > value = backendlib.load_function(BType, name) > - except KeyError: > - raise AttributeError(name) > + except KeyError, e: > + raise AttributeError('%s: %s' % (name, e)) > library.__dict__[name] = value > return > # > _______________________________________________ > pypy-commit mailing list > pypy-commit(a)python.org > http://mail.python.org/mailman/listinfo/pypy-commit -- Philip Jenvey

1 0

x is y <=> id(x)==id(y)
by Armin Rigo May 10, 2013

May 10, 2013

Hi all, I'm just wondering again about some "bug" reports that are not bugs, about people misusing "is" to compare two immutable objects. The current situation in PyPy is that "is" works like "==" for ints, longs, floats or complexes. It does not for strs or unicodes or tuples. Now of course someone on python-dev was (indirectly) complaining that you can compare in CPython ``x is ' '``, which works because single-character strings are cached, but not in PyPy. I'm sure someone else has been bitten by writing in CPython ``x is ()``, which is also cached there. (Fwiw I think that there is a design flaw somewhere in Python, to allow "1 is 1" to be executed without any error but also without any well-defined result...) Can we fix it once and for all? It's annoying because of id: if we want ``x is y`` for equal huge strings x and y, but still want ``id(x)==id(y)``, then we have to compute ``id(some_string)`` in a rather slow way, producing a huge number. The same for tuples: if we always want ``(1, 2) is (1, 2)`` then we need to compute ``id(some_tuple)`` recursively, which can also lead to huge numbers. In fact such a definition can explode the memory: ``a = (); for i in range(100): a = (a, a); id(a)`` would likely need a 2**100-digits number. Solution 2 would be to add these hacks specially for cases that CPython caches: I think by now we're only missing empty or single-char strings or unicodes, and empty tuple. Solution 3 would be to drop half of the rule, keeping only ``id(x)==id(y) => x is y``. This would be the easiest, as we could remove the complicated computations already done for longs or floats or complexes. We'd clearly document it as a difference from CPython. The question is what kind of code might break if we drop the case ``x is y => id(x)==id(y)``. A bientôt, Armin.

11 20