Mailman 3 November 2012 - pypy-dev

Python vs pypy: interesting performance difference [dict.setdefault]
by David Naylor 10 Mar '14

10 Mar '14

Hi, I needed to create a cache of date and time objects and I wondered what was the best way to handle the cache. For comparison I put together the following test: <file="iforkey.py"> import datetime import random import timeit ranges = [datetime.datetime(2011,01, random.randint(1, 31)) for i in xrange(1000)] def ifdict(): cache = {} b = [] for i in ranges: key = i.day if key in cache: b.append(cache[key]) else: date = i.date() cache[key] = date b.append(date) def keydict(): cache = {} b = [] for i in ranges: key = i.day try: b.append(cache[key]) except KeyError: date = i.date() cache[key] = date b.append(date) def defaultdict(): cache = {} b= [] for i in ranges: b.append(cache.setdefault(i, i.date())) print "ifdict:", timeit.repeat("ifdict()", "from __main__ import ifdict", number=10000) print "keydict:", timeit.repeat("keydict()", "from __main__ import keydict", number=10000) print "defaultdict:", timeit.repeat("defaultdict()", "from __main__ import defaultdict", number=10000) </file> # python iforfile.py ifdict: [2.432887077331543, 2.4002890586853027, 2.397233009338379] keydict: [2.3483030796051025, 2.358638048171997, 2.314802885055542] defaultdict: [3.5384328365325928, 3.5329859256744385, 3.5728111267089844] # pypy iforfile.py (pypy 1.5) ifdict: [0.8129069805145264, 0.74648118019104, 0.7432689666748047] keydict: [0.5187451839447021, 0.4662129878997803, 0.4504108428955078] defaultdict: [37.98510789871216, 37.859113931655884, 37.92770600318909] Pypy displays significant slowdown in the defaultdict function, otherwise displays its usual speedup. To check what is the cause I replaced i.date() with i.day and found no major difference in times. It appears dict.setdefault (or it's interaction with jit) is causing a slow down. Regards

4 14

numpy fails trigonometry with complex numbers, what to do?
by Matti Picus 22 Dec '13

22 Dec '13

I am trying to complete complex numbers in numpypy. Progress is good, I picked up from previous work on the numpypy-complex2 branch. Complex numbers come with extensive tests, it seems all the corner cases are covered. In porting the tests to numpypy, I came across a problem: numpy returns different results than cmath. Some of the differences are due to the fact that numpy does not raise a ValueError for dividing by 0 or other silly input values, but other differences are inexplicable (note the sign of the imaginary part): >>> numpy.arccos(complex(0.,-0.)) (1.5707963267948966-0j) >>> cmath.acos(complex(0.,-0.)) (1.5707963267948966+0j) >>> or this one: >>> cmath.acos(complex(float('inf'),2.3)) -infj >>> numpy.arccos(complex(float('inf'),2.3)) (0.78539816339744828-inf*j) Should I ignore the inconsistencies, or fix the 700 out of 2300 test instance failures? What should pypy's numpypy do - be consistent with numpy or with cmath? cmath is easier and probably faster (no need to mangle results or input args), so I would prefer cmath to trying to understand the logic behind numpy. Matti 2280c2365844

4 4

Re: [pypy-dev] [pypy-commit] pypy default: Do sign checks directly.
by Carl Friedrich Bolz 29 Dec '12

29 Dec '12

Hi Stian, Could the sign check be please put into a nice helper method on rbigint objects? I find it not so nice that longobject.py pokes around in the internals of the big integer implementation. Cheers, Carl Friedrich On 11/05/2012 03:43 PM, Stian Andreassen wrote: > Author: Stian Andreassen > Branch: > Changeset: r58761:bc1b37bec5b3 > Date: 2012-11-05 23:41 +0100 > http://bitbucket.org/pypy/pypy/changeset/bc1b37bec5b3/ > > Log: Do sign checks directly. > > diff --git a/pypy/objspace/std/longobject.py b/pypy/objspace/std/longobject.py > --- a/pypy/objspace/std/longobject.py > +++ b/pypy/objspace/std/longobject.py > @@ -251,7 +251,7 @@ > > def pow__Long_Long_Long(space, w_long1, w_long2, w_long3): > # XXX need to replicate some of the logic, to get the errors right > - if w_long2.num.lt(rbigint.fromint(0)): > + if w_long2.num.sign < 0: > raise OperationError( > space.w_TypeError, > space.wrap( > @@ -265,7 +265,7 @@ > > def pow__Long_Long_None(space, w_long1, w_long2, w_long3): > # XXX need to replicate some of the logic, to get the errors right > - if w_long2.num.lt(rbigint.fromint(0)): > + if w_long2.num.sign < 0: > raise FailedToImplementArgs( > space.w_ValueError, > space.wrap("long pow() too negative")) > @@ -288,7 +288,7 @@ > > def lshift__Long_Long(space, w_long1, w_long2): > # XXX need to replicate some of the logic, to get the errors right > - if w_long2.num.lt(rbigint.fromint(0)): > + if w_long2.num.sign < 0: > raise OperationError(space.w_ValueError, > space.wrap("negative shift count")) > try: > @@ -300,7 +300,7 @@ > > def rshift__Long_Long(space, w_long1, w_long2): > # XXX need to replicate some of the logic, to get the errors right > - if w_long2.num.lt(rbigint.fromint(0)): > + if w_long2.num.sign < 0: > raise OperationError(space.w_ValueError, > space.wrap("negative shift count")) > try: > _______________________________________________ > pypy-commit mailing list > pypy-commit(a)python.org > http://mail.python.org/mailman/listinfo/pypy-commit >

7 28

Translation PDF link is broken
by anatoly techtonik 28 Nov '12

28 Nov '12

This page - http://doc.pypy.org/en/latest/translation.html - links to http://doc.pypy.org/en/latest/image/translation.pdf which is 404. If you fixing that - is it possible to link PyGame viewer to more information as well? -- anatoly t.

3 7

PyPy 2.0 beta 1 released
by Maciej Fijalkowski 25 Nov '12

25 Nov '12

We're pleased to announce the 2.0 beta 1 release of PyPy. This release is not a typical beta, in a sense the stability is the same or better than 1.9 and can be used in production. It does however include a few performance regressions documented below that don't allow us to label is as 2.0 final. (It also contains many performance improvements.) The main features of this release are support for ARM processor and compatibility with CFFI. It also includes numerous improvements to the numpy in pypy effort, cpyext and performance. You can download the PyPy 2.0 beta 1 release here: http://pypy.org/download.html What is PyPy? ============= PyPy is a very compliant Python interpreter, almost a drop-in replacement for CPython 2.7.3. It's fast (`pypy 2.0 beta 1 and cpython 2.7.3`_ performance comparison) due to its integrated tracing JIT compiler. This release supports x86 machines running Linux 32/64, Mac OS X 64 or Windows 32. It also supports ARM machines running Linux. Windows 64 work is still stalling, we would welcome a volunteer to handle that. .. _`pypy 2.0 beta 1 and cpython 2.7.3`: http://bit.ly/USXqpP How to use PyPy? ================ We suggest using PyPy from a `virtualenv`_. Once you have a virtualenv installed, you can follow instructions from `pypy documentation`_ on how to proceed. This document also covers other `installation schemes`_. .. _`pypy documentation`: http://doc.pypy.org/en/latest/getting-started.html#installing-using-virtual… .. _`virtualenv`: http://www.virtualenv.org/en/latest/ .. _`installation schemes`: http://doc.pypy.org/en/latest/getting-started.html#installing-pypy .. _`PyPy and pip`: http://doc.pypy.org/en/latest/getting-started.html#installing-pypy Regressions =========== Reasons why this is not PyPy 2.0: * the ``ctypes`` fast path is now slower than it used to be. In PyPy 1.9 ``ctypes`` was either incredibly faster or slower than CPython depending whether you hit the fast path or not. Right now it's usually simply slower. We're probably going to rewrite ``ctypes`` using ``cffi``, which will make it universally faster. * ``cffi`` (an alternative to interfacing with C code) is very fast, but it is missing one optimization that will make it as fast as a native call from C. * ``numpypy`` lazy computation was disabled for the sake of simplicity. We should reenable this for the final 2.0 release. Highlights ========== * ``cffi`` is officially supported by PyPy. You can install it normally by using ``pip install cffi`` once you have installed `PyPy and pip`_. The corresponding ``0.4`` version of ``cffi`` has been released. * ARM is now an officially supported processor architecture. PyPy now work on soft-float ARM/Linux builds. Currently ARM processors supporting the ARMv7 and later ISA that include a floating-point unit are supported. * This release contains the latest Python standard library 2.7.3 and is fully compatible with Python 2.7.3. * It does not however contain hash randomization, since the solution present in CPython is not solving the problem anyway. The reason can be found on the `CPython issue tracker`_. * ``gc.get_referrers()`` is now faster. * Various numpy improvements. The list includes: * axis argument support in many places * full support for fancy indexing * ``complex128`` and ``complex64`` dtypes * `JIT hooks`_ are now a powerful tool to introspect the JITting process that PyPy performs. * ``**kwds`` usage is much faster in the typical scenario * operations on ``long`` objects are now as fast as in CPython (from roughly 2x slower) * We now have special strategies for ``dict``/``set``/``list`` which contain unicode strings, which means that now such collections will be both faster and more compact. .. _`cpython issue tracker`: http://bugs.python.org/issue14621 .. _`jit hooks`: http://doc.pypy.org/en/latest/jit-hooks.html Things we're working on ======================= There are a few things that did not make it to the 2.0 beta 1, which are being actively worked on. Greenlets support in the JIT is one that we would like to have before 2.0 final. Two important items that will not make it to 2.0, but are being actively worked on, are: * Faster JIT warmup time. * Software Transactional Memory. Cheers, Maciej Fijalkowski, Armin Rigo and the PyPy team

3 4

Surface-level Presentation about PyPy
by Dan Stromberg 24 Nov '12

24 Nov '12

There's been some curiosity about PyPy at my local Python User Group (OCPUG), so I put together a brief, surface-level talk about PyPy, to be delivered on Tuesday evening (the 27th). If some of the more PyPy-knowledgeable people here could look it over for accuracy and omissions, that'd be awesome. It's #4 on the URL below: http://stromberg.dnsalias.org/~dstromberg/Python-Talks/ As I currently see it, the main omission at this point is how to install PyPy after building one from mercurial. I have a script I've been using for this, but I suspect there's a more official way of doing it that would be better for sharing with others. Thanks!

2 1

Need help disutils module with PyPy
by Sasikanth Eda 23 Nov '12

23 Nov '12

Hi All, I am a newbie facing problem with utilization of disutils ( http://docs.python.org/2/distutils/introduction.html) module that helps in distributing python modules with PyPy. *content of demo.c:* #include <Python.h> #include <stdio.h> int main (void) { printf("setup passed\n"); return 0; } *content of setup.py:* from distutils.core import setup, Extension module1 = Extension('demo', sources = ['demo.c']) setup (name = 'PackageName', version = '1.0', description = 'This is a demo package', ext_modules = [module1]) Now the problem is ; if this setup.py is compiled with Python, then I am able to successfully generate the '.so' [linux@localhost disutils_example]$ python setup.py build running build running build_ext building 'demo' extension creating build creating build/temp.linux-x86_64-2.7 gcc -pthread -fno-strict-aliasing -O2 -g -pipe -Wall -Wp,-D_FORTIFY_SOURCE=2 -fexceptions -fstack-protector --param=ssp-buffer-size=4 -m64 -mtune=generic -D_GNU_SOURCE -fPIC -fwrapv -DNDEBUG -O2 -g -pipe -Wall -Wp,-D_FORTIFY_SOURCE=2 -fexceptions -fstack-protector --param=ssp-buffer-size=4 -m64 -mtune=generic -D_GNU_SOURCE -fPIC -fwrapv -fPIC -I/usr/include/python2.7 -c demo.c -o build/temp.linux-x86_64-2.7/demo.o creating build/lib.linux-x86_64-2.7 gcc -pthread -shared -Wl,-z,relro build/temp.linux-x86_64-2.7/demo.o -L/usr/lib64 -lpython2.7 -o build/lib.linux-x86_64-2.7/demo.so where as using PyPy, i am getting the following error [linux@localhost disutils_example]$ pypy setup.py build running build running build_ext building 'demo' extension cc -fPIC -Wimplicit -I/usr/lib64/pypy-1.9/include -c demo.c -o build/temp.linux-x86_64-2.7/demo.o demo.c:1:20: fatal error: Python.h: No such file or directory compilation terminated. error: command 'cc' failed with exit status 1 where as python-devel package which contains these header files is installed. Kindly suggest me the process to make this compile with PyPy. Thanking you, Sasikanth

2 1

problem with twisted in current pypy master
by Max Lavrenov 23 Nov '12

23 Nov '12

Hello everyone I had got this error while installed twisted. I just built pypy from master, made virtual enviroment and pip install twisted. Traceback (most recent call last): File "app_main.py", line 51, in run_toplevel File "app_main.py", line 547, in run_it File "<string>", line 1, in <module> File "/home/e-max/.virtualenvs/pypy/build/twisted/setup.py", line 82, in <module> main(sys.argv[1:]) File "/home/e-max/.virtualenvs/pypy/build/twisted/setup.py", line 77, in main setup(**setup_args) File "./twisted/python/dist.py", line 58, in setup return core.setup(**get_setup_args(**kw)) File "/home/e-max/workspace/pypy/lib-python/2.7/distutils/core.py", line 152, in setup dist.run_commands() File "/home/e-max/workspace/pypy/lib-python/2.7/distutils/dist.py", line 953, in run_commands self.run_command(cmd) File "/home/e-max/workspace/pypy/lib-python/2.7/distutils/dist.py", line 972, in run_command cmd_obj.run() File "/home/e-max/.virtualenvs/pypy/site-packages/setuptools-0.6c11-py2.7.egg/setuptools/command/install.py", line 56, in run return _install.run(self) File "/home/e-max/workspace/pypy/lib-python/2.7/distutils/command/install.py", line 584, in run self.run_command(cmd_name) File "/home/e-max/workspace/pypy/lib-python/2.7/distutils/cmd.py", line 326, in run_command self.distribution.run_command(command) File "/home/e-max/workspace/pypy/lib-python/2.7/distutils/dist.py", line 972, in run_command cmd_obj.run() File "/home/e-max/.virtualenvs/pypy/site-packages/setuptools-0.6c11-py2.7.egg/setuptools/command/install_lib.py", line 24, in run self.byte_compile(outfiles) File "/home/e-max/workspace/pypy/lib-python/2.7/distutils/command/install_lib.py", line 138, in byte_compile dry_run=self.dry_run) File "/home/e-max/workspace/pypy/lib-python/2.7/distutils/util.py", line 559, in byte_compile compile(file, cfile, dfile) File "/home/e-max/workspace/pypy/lib-python/2.7/py_compile.py", line 126, in compile marshal.dump(codeobject, fc) UnicodeEncodeError: 'utf-8' codec can't encode character u'\udf42' in position 0: surrogates not allowed Best regards, Max

3 4

New Contributor Experience in Python and other FOSS Communities - A Survey
by Brian Curtin 19 Nov '12

19 Nov '12

Hi all, [note - this was originally sent to a few CPython places, but I told the surveyor that we should include the many new PyPy contributors as well, and he agreed] Along with a number of other free and open communities, Python is being included in a survey of new contributors since January 2010. The survey is being done by Kevin Carillo, a PhD student at Victoria University of Wellington who is studying various approaches that projects use to gain and work with new contributors. If you have written patches, reviewed issues, or done anything to contribute to the development of Python and you started this after January 2010, I hope you can spare around 20 minutes to complete this survey: https://limesurvey.sim.vuw.ac.nz/index.php?sid=65151&lang=en There's a longer blog post up at http://blog.python.org/2012/11/new-contributor-experience-in-python.html if you would like a bit more information. On behalf of Kevin Carillo, I thank you for your time and consideration of this survey. Thank, Brian Curtin

1 0

Re: [pypy-dev] [pypy-commit] pypy default: INSTANCE_PTR_{EQ, NE} also do not escape their arguments in the JIT
by Carl Friedrich Bolz 16 Nov '12

16 Nov '12

Hi Alex, Please write a test for this change. Cheers, Carl Friedrich alex_gaynor <noreply(a)buildbot.pypy.org> wrote: >Author: Alex Gaynor <alex.gaynor(a)gmail.com> >Branch: >Changeset: r58945:e72d4f5b720e >Date: 2012-11-16 07:11 -0800 >http://bitbucket.org/pypy/pypy/changeset/e72d4f5b720e/ > >Log: INSTANCE_PTR_{EQ,NE} also do not escape their arguments in the JIT > >diff --git a/pypy/jit/metainterp/heapcache.py >b/pypy/jit/metainterp/heapcache.py >--- a/pypy/jit/metainterp/heapcache.py >+++ b/pypy/jit/metainterp/heapcache.py >@@ -74,7 +74,9 @@ > elif (opnum != rop.GETFIELD_GC and > opnum != rop.MARK_OPAQUE_PTR and > opnum != rop.PTR_EQ and >- opnum != rop.PTR_NE): >+ opnum != rop.PTR_NE and >+ opnum != rop.INSTANCE_PTR_EQ and >+ opnum != rop.INSTANCE_PTR_NE): > idx = 0 > for box in argboxes: > # setarrayitem_gc don't escape its first argument >_______________________________________________ >pypy-commit mailing list >pypy-commit(a)python.org >http://mail.python.org/mailman/listinfo/pypy-commit Carl Friedrich Bolz

1 0

pypy-dev November 2012