Mailman 3 October 2014 - pypy-dev

use as benchmark pypy vs python if you please
by Dima Tisnek 13 Sep '15

13 Sep '15

Attached is n-queens solver (pardon my naive algorithm), it runs: python 2.7.6: 17s pypy 2.4.0 alpha: 23s same nojit: 32s I've tried similar-looking algorithm for another problem before, and has similar results -- somehow pypy was slower. feel free to investigate / tweak or even use on speed.pypy.org

4 4

Silly Question re PIP
by Gary Furash 01 Nov '14

01 Nov '14

I didn't understand the documentation, but is it the case that you cannot use PIP with pypy? When I run pypy get_pip.py, it fails. -- gary furash | furashgary(a)gmail.com, 520-907-2470 | calendar <https://www.google.com/calendar/embed?src=furashgary%40gmail.com&ctz=Americ…>

4 3

cppyy on Ubuntu 14.04
by exarkun＠twistedmatrix.com 31 Oct '14

31 Oct '14

Hello, I've been looking at cppyy a bit (with the goal of using an existing project based on it). I haven't gotten far though. When I try to use cppyy with the Ubuntu-packaged PyPy (on Ubuntu 14.04), I get this: Python 2.7.3 (2.2.1+dfsg-1, Nov 28 2013, 05:13:10) [PyPy 2.2.1 with GCC 4.8.2] on linux2 Type "help", "copyright", "credits" or "license" for more information. And now for something completely different: ``Is it a cactus bug or problem with my war?'' >>>> import cppyy Traceback (most recent call last): File "<stdin>", line 1, in <module> ImportError: missing reflection library libcppyy_backend.so >>>> I see that this is a somewhat old version of PyPy - but is that the problem? Thanks, Jean-Paul

4 5

Short Term Investment Saving Plan
by MISS.JOLIYA 30 Oct '14

30 Oct '14

"PROSPECTING" - - - - - - - - - - - - - - Dear sir, state life has introduced a very unique short term investment plan with very attractive feature With the surety of Govt. Of Pakistan for Age Group 25 to 45 years. If you are interested to see this plan, send your E - mail address with your Basic information . 1. Full Name. 2. Your Date Of Birth 3. Marital Status; 4. Child Age 5. Wife Age 6. Your Income/Salary 7. Your Contact No 8. Location Mohammad. Hanif. Kasbati. STATE LIFE INSURANCE CROP OF PAKISTAN SALES MANAGER, K C Z, CELL NO. 0333-218-9282 OFFICE. 0092-21-32471496 SKYPE ID: m.hanif.kasbati E. Mail. Id. mohammadhanifkasbati(a)yahoo.com

1 0

Re: [pypy-dev] PyPy3 2.4.0 released
by Philip Jenvey 30 Oct '14

30 Oct '14

On Oct 24, 2014, at 3:02 PM, Phyo Arkar <phyo.arkarlwin(a)gmail.com> wrote: > Congrats! > I am using Pypy 2.4 for tornado based web development , performance > improvement is the real deal there. > > i am waiting to switch to python3 and its becoming near! Its when > pypy3 3.4 is ready (ie , Python3 is only starting to interesting after > 3.4.x due to asyncio module) . > > I have a question: > In CPython , python 2 vs python 3 is around 10-30% performance > difference. In PyPy world is there such difference? Same performance > between pypy2 and pypy3 ? Hi there, PyPy3 performance isn’t quite as good as PyPy2’s yet. Our main focus has been on compatibility, as you said yourself, you probably won’t switch until a later version of Python 3’s supported. The benchmark infrastructure for PyPy3 is in need of help. Pypy's own benchmark suite used by http://speed.pypy.org isn't Python 3 compatible yet, nor is the site really setup to run PyPy3 benchmark runs. A few years ago there were plans for the creation of a shared between VMs/the PSF version of speed.pypy.org (to be http://speed.python.org), unfortunately these efforts seemed to have failed or at the very least stalled indefinitely. However the benchmark suite intended for this new site supports Python 3. It’s similar to pypy’s own suite, albeit with a few different benchmarks and unfortunately separately maintained. Here’s output from a run of it with CPython 3.2.5 vs PyPy3 2.4.0: though this was ran on a laptop which isn’t going to produce the most stable numbers but it'll give you some idea: ### 2to3 ### 8.652382 -> 9.627286: 1.11x slower ### call_method ### Min: 0.385220 -> 0.011149: 34.55x faster Avg: 0.412039 -> 0.012474: 33.03x faster Significant (t=293.52) Stddev: 0.01644 -> 0.00279: 5.8934x smaller ### call_method_slots ### Min: 0.394147 -> 0.011669: 33.78x faster Avg: 0.419456 -> 0.013503: 31.06x faster Significant (t=185.89) Stddev: 0.02655 -> 0.00326: 8.1550x smaller ### call_method_unknown ### Min: 0.416863 -> 0.016139: 25.83x faster Avg: 0.444017 -> 0.019560: 22.70x faster Significant (t=187.44) Stddev: 0.02645 -> 0.00835: 3.1669x smaller ### call_simple ### Min: 0.324141 -> 0.011182: 28.99x faster Avg: 0.343808 -> 0.013248: 25.95x faster Significant (t=235.96) Stddev: 0.01667 -> 0.00406: 4.1072x smaller ### chameleon ### Min: 0.061195 -> 0.010301: 5.94x faster Avg: 0.066450 -> 0.015595: 4.26x faster Significant (t=52.46) Stddev: 0.00668 -> 0.00982: 1.4710x larger ### chaos ### Min: 0.340240 -> 0.011240: 30.27x faster Avg: 0.366752 -> 0.018671: 19.64x faster Significant (t=60.31) Stddev: 0.01713 -> 0.03704: 2.1620x larger ### django_v2 ### Min: 0.692480 -> 0.029994: 23.09x faster Avg: 0.731315 -> 0.037337: 19.59x faster Significant (t=173.98) Stddev: 0.02693 -> 0.00837: 3.2160x smaller ### etree_generate ### Min: 0.599923 -> 1.087517: 1.81x slower Avg: 0.636505 -> 1.149231: 1.81x slower Significant (t=-77.16) Stddev: 0.02478 -> 0.03993: 1.6114x larger ### etree_parse ### Min: 0.296519 -> 0.931630: 3.14x slower Avg: 0.331876 -> 1.010314: 3.04x slower Significant (t=-78.30) Stddev: 0.03799 -> 0.04807: 1.2652x larger ### etree_process ### Min: 0.421074 -> 0.739647: 1.76x slower Avg: 0.454918 -> 0.798387: 1.76x slower Significant (t=-55.61) Stddev: 0.03127 -> 0.03048: 1.0260x smaller ### fannkuch ### Min: 1.744042 -> 0.179821: 9.70x faster Avg: 1.815167 -> 0.193547: 9.38x faster Significant (t=293.69) Stddev: 0.03619 -> 0.01465: 2.4707x smaller ### fastpickle ### Min: 0.671650 -> 109.014578: 162.31x slower Avg: 0.708033 -> 115.379135: 162.96x slower Significant (t=-299.42) Stddev: 0.02026 -> 2.70799: 133.6909x larger ### fastunpickle ### Min: 0.586665 -> 55.736337: 95.01x slower Avg: 0.625024 -> 56.833369: 90.93x slower Significant (t=-357.75) Stddev: 0.02488 -> 1.11069: 44.6438x larger ### float ### Min: 0.358756 -> 0.026437: 13.57x faster Avg: 0.387649 -> 0.041561: 9.33x faster Significant (t=106.89) Stddev: 0.02159 -> 0.00762: 2.8351x smaller ### formatted_logging ### Min: 0.306111 -> 0.086534: 3.54x faster Avg: 0.331045 -> 0.116844: 2.83x faster Significant (t=32.06) Stddev: 0.02016 -> 0.04273: 2.1198x larger ### go ### Min: 0.640692 -> 0.110813: 5.78x faster Avg: 0.686035 -> 0.153473: 4.47x faster Significant (t=81.23) Stddev: 0.02366 -> 0.03987: 1.6853x larger ### hexiom2 ### Min: 168.499830 -> 13.845624: 12.17x faster Avg: 170.731699 -> 14.320609: 11.92x faster Significant (t=68.55) Stddev: 3.15634 -> 0.67173: 4.6988x smaller ### iterative_count ### Min: 0.155776 -> 0.001828: 85.22x faster Avg: 0.169719 -> 0.001959: 86.63x faster Significant (t=66.83) Stddev: 0.01775 -> 0.00017: 104.6217x smaller ### json_dump ### Min: 0.563337 -> 2.088108: 3.71x slower Avg: 0.600209 -> 2.212642: 3.69x slower Significant (t=-92.38) Stddev: 0.01963 -> 0.12185: 6.2066x larger ### json_dump_v2 ### Min: 3.703154 -> 12.685391: 3.43x slower Avg: 3.906239 -> 13.462007: 3.45x slower Significant (t=-73.61) Stddev: 0.16381 -> 0.90320: 5.5139x larger ### json_load ### Min: 0.637112 -> 0.284530: 2.24x faster Avg: 0.691114 -> 0.340403: 2.03x faster Significant (t=48.13) Stddev: 0.03443 -> 0.03834: 1.1134x larger ### mako ### Min: 0.068892 -> 0.023433: 2.94x faster Avg: 0.073817 -> 0.034535: 2.14x faster Significant (t=45.74) Stddev: 0.01044 -> 0.00868: 1.2032x smaller ### mako_v2 ### Min: 0.055735 -> 0.023094: 2.41x faster Avg: 0.060464 -> 0.034531: 1.75x faster Significant (t=39.36) Stddev: 0.01003 -> 0.01079: 1.0764x larger ### meteor_contest ### Min: 0.234554 -> 0.101157: 2.32x faster Avg: 0.253522 -> 0.112062: 2.26x faster Significant (t=45.33) Stddev: 0.01735 -> 0.01364: 1.2717x smaller ### nbody ### Min: 0.332543 -> 0.024746: 13.44x faster Avg: 0.352898 -> 0.026261: 13.44x faster Significant (t=111.01) Stddev: 0.02059 -> 0.00298: 6.9071x smaller ### normal_startup ### Min: 0.974583 -> 1.578062: 1.62x slower Avg: 1.011902 -> 1.640190: 1.62x slower Significant (t=-144.15) Stddev: 0.01753 -> 0.02535: 1.4458x larger ### nqueens ### Min: 0.357568 -> 0.052418: 6.82x faster Avg: 0.386644 -> 0.055879: 6.92x faster Significant (t=98.78) Stddev: 0.02223 -> 0.00816: 2.7244x smaller ### pathlib ### Min: 0.119940 -> 0.384461: 3.21x slower Avg: 0.130969 -> 0.504688: 3.85x slower Significant (t=-142.33) Stddev: 0.01389 -> 0.03912: 2.8161x larger ### pickle_dict ### Min: 0.496007 -> 8.778371: 17.70x slower Avg: 0.522927 -> 9.051905: 17.31x slower Significant (t=-306.08) Stddev: 0.01866 -> 0.19615: 10.5126x larger ### pickle_list ### Min: 0.295601 -> 6.132281: 20.75x slower Avg: 0.315390 -> 6.327789: 20.06x slower Significant (t=-215.11) Stddev: 0.01909 -> 0.19672: 10.3048x larger ### pidigits ### Min: 0.345745 -> 0.297060: 1.16x faster Avg: 0.362924 -> 0.316708: 1.15x faster Significant (t=13.14) Stddev: 0.01719 -> 0.01798: 1.0456x larger ### raytrace ### Min: 1.527975 -> 0.046997: 32.51x faster Avg: 1.575889 -> 0.052239: 30.17x faster Significant (t=344.42) Stddev: 0.02992 -> 0.00911: 3.2830x smaller ### regex_compile ### Min: 0.462712 -> 0.076845: 6.02x faster Avg: 0.489184 -> 0.159849: 3.06x faster Significant (t=11.67) Stddev: 0.01954 -> 0.19853: 10.1609x larger ### regex_effbot ### Min: 0.061423 -> 0.034389: 1.79x faster Avg: 0.065657 -> 0.041867: 1.57x faster Significant (t=16.14) Stddev: 0.00658 -> 0.00808: 1.2285x larger ### richards ### Min: 0.201109 -> 0.004319: 46.56x faster Avg: 0.215034 -> 0.004968: 43.28x faster Significant (t=91.42) Stddev: 0.01619 -> 0.00140: 11.5878x smaller ### silent_logging ### Min: 0.080098 -> 0.007613: 10.52x faster Avg: 0.085028 -> 0.009930: 8.56x faster Significant (t=43.10) Stddev: 0.00980 -> 0.00747: 1.3113x smaller ### simple_logging ### Min: 0.296057 -> 0.081953: 3.61x faster Avg: 0.315012 -> 0.110826: 2.84x faster Significant (t=33.22) Stddev: 0.01866 -> 0.03925: 2.1031x larger ### spectral_norm ### Min: 0.365987 -> 0.016084: 22.75x faster Avg: 0.393062 -> 0.018651: 21.07x faster Significant (t=124.58) Stddev: 0.02014 -> 0.00679: 2.9647x smaller ### startup_nosite ### Min: 0.485740 -> 0.613260: 1.26x slower Avg: 0.518384 -> 0.654354: 1.26x slower Significant (t=-49.49) Stddev: 0.01842 -> 0.02038: 1.1066x larger ### telco ### Min: 0.773569 -> 0.043440: 17.81x faster Avg: 0.800002 -> 0.049813: 16.06x faster Significant (t=276.68) Stddev: 0.01300 -> 0.01409: 1.0844x larger ### threaded_count ### Min: 0.167072 -> 0.005613: 29.76x faster Avg: 0.193020 -> 0.005944: 32.47x faster Significant (t=88.19) Stddev: 0.01499 -> 0.00036: 41.4676x smaller ### tornado_http ### Min: 0.307076 -> 0.390125: 1.27x slower Avg: 0.378676 -> 0.459946: 1.21x slower Significant (t=-8.21) Stddev: 0.04109 -> 0.09005: 2.1916x larger ### unpack_sequence ### Min: 0.000052 -> 0.000057: 1.10x slower Avg: 0.000077 -> 0.000404: 5.21x slower Significant (t=-16.30) Stddev: 0.00011 -> 0.00447: 41.4670x larger ### unpickle_list ### Min: 0.455635 -> 0.963103: 2.11x slower Avg: 0.514364 -> 1.017570: 1.98x slower Significant (t=-79.98) Stddev: 0.02640 -> 0.03581: 1.3564x larger So PyPy3’s generally faster, except on startup and tornado_http, where it’s only slightly slower. Then it’s quite a bit slower on “mini interpreters” such as etree/json/Pickle. These modules also happen to be pure/mostly pure python versions on PyPy vs C versions used by CPython. Pickle looks particularly bad, I have a guess that it’s due to removal of an optimization in marshal that happened during py3k compatibility work, see: https://bitbucket.org/pypy/pypy/src/a245775c1e3687662879fe57964a5ece689cd78… and it’s still slower than PyPy2 on many things. Anyway, to reiterate we’ve pretty busy with compatibility work. We could use some help with benchmarking infrastructure =] -- Philip Jenvey

2 1

Weird Unicode errors
by Timothy Baldridge 27 Oct '14

27 Oct '14

My interpreter is built using mostly unicode for symbols and strings, but recently I've been getting some really weird translation errors. The first is this: https://gist.github.com/halgari/0d57dd87434968561705 I tracked this error down to being caused whenever I try an isinstance of unicode like this: isinstance(x, unicode) This is really annoying as I'd love to have a single unified wrap function: @specialize.argtype(0) def wrap(x): if isinstance(x, int): return numbers.Integer(x) if isinstance(x, unicode): return String(x) And as of this morning I started getting errors like this: [translation:ERROR] TyperError: don't know how to convert from <UnicodeRepr * GcStruct rpy_unicode { hash, chars }> to <UniCharRepr UniChar> [translation:ERROR] .. (pixie.vm.reader:47)PromptReader.read [translation:ERROR] .. block@82 with 1 exits [translation:ERROR] .. v235 = simple_call(v234) What is a UniChar? My code only deals with unicode strings, so I'm not sure what's happening here. Thanks for any help. I've had unicode working perfectly with my interpreter for weeks, and suddenly in the past two days I've started getting these errors. Timothy

2 3

translation error on jit.unroll_safe with jit_libffi
by Timothy Baldridge 25 Oct '14

25 Oct '14

I'm in the process of adding FFI to pixie. It translates and runs fine (I'm sure there's still bugs). But the trace is a bit verbose, so I decided I'd unroll the loop that packs the arguments in the exchange buffer. However then I get this error: [translation:info] File "<388-codegen /Users/tim/oss-not-dropbox/externals/pypy/rpython/rtyper/rtyper.py:540>", line 4, in translate_op_call_args [translation:info] return r_arg1.rtype_call_args(hop) [translation:info] File "/Users/tim/oss-not-dropbox/externals/pypy/rpython/rtyper/rpbc.py", line 659, in rtype_call_args [translation:info] return self.redispatch_call(hop, call_args=True) [translation:info] File "/Users/tim/oss-not-dropbox/externals/pypy/rpython/rtyper/rpbc.py", line 686, in redispatch_call [translation:info] hop.llops, hop) [translation:info] File "/Users/tim/oss-not-dropbox/externals/pypy/rpython/rtyper/rclass.py", line 1075, in rtype_new_instance [translation:info] return rinstance.new_instance(llops, classcallhop) [translation:info] File "/Users/tim/oss-not-dropbox/externals/pypy/rpython/rtyper/rclass.py", line 715, in new_instance [translation:info] r.convert_desc_or_const(value)) [translation:info] File "/Users/tim/oss-not-dropbox/externals/pypy/rpython/rtyper/rmodel.py", line 115, in convert_desc_or_const [translation:info] return self.convert_const(desc_or_const.value) [translation:info] File "/Users/tim/oss-not-dropbox/externals/pypy/rpython/rtyper/lltypesystem/rpbc.py", line 158, in convert_const [translation:info] funcdesc = self.rtyper.annotator.bookkeeper.getdesc(value) [translation:info] File "/Users/tim/oss-not-dropbox/externals/pypy/rpython/annotator/bookkeeper.py", line 396, in getdesc [translation:info] raise Exception("%s: %r" % (msg, pyobj)) [translation:ERROR] Exception: unexpected prebuilt constant: <staticmethod object at 0x10a8d5bb0> The line I'm un-commenting is here: https://github.com/pixie-lang/pixie/blob/master/pixie/vm/libs/ffi.py#L150 >From what I can tell this bug is from the translator trying to do something with the static methods in CallDescr. Why this is happening, I haven't a clue. Timothy

4 8

PyPy3 2.4.0 released
by Philip Jenvey 24 Oct '14

24 Oct '14

================================================= PyPy3 2.4 - Snow White ================================================= We're pleased to announce PyPy3 2.4, which contains significant performance enhancements and bug fixes. You can download the PyPy3 2.4.0 release here: http://pypy.org/download.html PyPy3 Highlights ================ Issues reported with our previous release were fixed after reports from users on our new issue tracker at https://bitbucket.org/pypy/pypy/issues or on IRC at #pypy. Here is a summary of the user-facing PyPy3 specific changes: * Better Windows compatibility, e.g. the nt module functions _getfinalpathname & _getfileinformation are now supported (the former is required for the popular pathlib library for example) * Various fsencode PEP 383 related fixes to the posix module (readlink, uname, ttyname and ctermid) and improved locale handling * Switched default binary name os POSIX distributions to 'pypy3' (which symlinks to to 'pypy3.2') * Fixed a couple different crashes related to parsing Python 3 source code Further Highlights (shared w/ PyPy2) ==================================== Benchmarks improved after internal enhancements in string and bytearray handling, and a major rewrite of the GIL handling. This means that external calls are now a lot faster, especially the CFFI ones. It also means better performance in a lot of corner cases with handling strings or bytearrays. The main bugfix is handling of many socket objects in your program which in the long run used to "leak" memory. We fixed a memory leak in IO in the sandbox_ code We welcomed more than 12 new contributors, and conducted two Google Summer of Code projects, as well as other student projects not directly related to Summer of Code. * Reduced internal copying of bytearray operations * Tweak the internal structure of StringBuilder to speed up large string handling, which becomes advantageous on large programs at the cost of slightly slower small *benchmark* type programs. * Boost performance of thread-local variables in both unjitted and jitted code, this mostly affects errno handling on linux, which makes external calls faster. * Move to a mixed polling and mutex GIL model that make mutlithreaded jitted code run *much* faster * Optimize errno handling in linux (x86 and x86-64 only) * Remove ctypes pythonapi and ctypes.PyDLL, which never worked on PyPy * Classes in the ast module are now distinct from structures used by the compiler, which simplifies and speeds up translation of our source code to the PyPy binary interpreter * Win32 now links statically to zlib, expat, bzip, and openssl-1.0.1i. No more missing DLLs * Many issues were resolved_ since the 2.3.1 release in June .. _`whats-new`: http://doc.pypy.org/en/latest/whatsnew-2.4.0.html .. _resolved: https://bitbucket.org/pypy/pypy/issues?status=resolved .. _sandbox: http://doc.pypy.org/en/latest/sandbox.html We have further improvements on the way: rpython file handling, numpy linalg compatibility, as well as improved GC and many smaller improvements. Please try it out and let us know what you think. We especially welcome success stories, we know you are using PyPy, please tell us about it! Cheers The PyPy Team

2 1

PyPy Warsaw Sprint (October 21-25th, 2014)
by Valentina Mukhamedzhanova 20 Oct '14

20 Oct '14

Hi, I'm in Warsaw these days and I learned that the next PyPy sprint is taking place here, so I'd like to attend it on October 21st. I participated in the PyPy sprint at EuroPython this year. I don't have any specific task to work on, will probably try to fix some missing 3.3 support. Valentina

2 1

Please update the buildbot
by Philip Jenvey 19 Oct '14

19 Oct '14

Could someone with access update and cycle the buildbot when they get the chance? Thanks, -- Philip Jenvey

6 9