Mailman 3 August 2013 - pypy-dev

Cross-post from python-ideas: Compressing the stack on the fly
by Ram Rachum 28 Jul '14

28 Jul '14

Hi guys, I made a post on the Python-ideas mailing list that I was told might be relevant to Pypy. I've reproduced the original email below. Here is the thread on Python-ideas with all the discussion.<https://groups.google.com/forum/?fromgroups#!topic/python-ideas/hteGSNTyC_4> -------------- Hi everybody, Here's an idea I had a while ago. Now, I'm an ignoramus when it comes to how programming languages are implemented, so this idea will most likely be either (a) completely impossible or (b) trivial knowledge. I was thinking about the implementation of the factorial in Python. I was comparing in my mind 2 different solutions: The recursive one, and the one that uses a loop. Here are example implementations for them: def factorial_recursive(n): if n == 1: return 1 return n * factorial_recursive(n - 1) def factorial_loop(n): result = 1 for i in range(1, n + 1): result *= i return result I know that the recursive one is problematic, because it's putting a lot of items on the stack. In fact it's using the stack as if it was a loop variable. The stack wasn't meant to be used like that. Then the question came to me, why? Maybe the stack could be built to handle this kind of (ab)use? I read about tail-call optimization on Wikipedia. If I understand correctly, the gist of it is that the interpreter tries to recognize, on a frame-by-frame basis, which frames could be completely eliminated, and then it eliminates those. Then I read Guido's blog post explaining why he doesn't want it in Python. In that post he outlined 4 different reasons why TCO shouldn't be implemented in Python. But then I thought, maybe you could do something smarter than eliminating individual stack frames. Maybe we could create something that is to the current implementation of the stack what `xrange` is to the old-style `range`. A smart object that allows access to any of a long list of items in it, without actually having to store those items. This would solve the first argument that Guido raises in his post, which I found to be the most substantial one. What I'm saying is: Imagine the stack of the interpreter when it runs the factorial example above for n=1000. It has around 1000 items in it and it's just about to explode. But then, if you'd look at the contents of that stack, you'd see it's embarrassingly regular, a compression algorithm's wet dream. It's just the same code location over and over again, with a different value for `n`. So what I'm suggesting is an algorithm to compress that stack on the fly. An algorithm that would detect regularities in the stack and instead of saving each individual frame, save just the pattern. Then, there wouldn't be any problem with showing informative stack trace: Despite not storing every individual frame, each individual frame could still be accessed, similarly to how `xrange` allow access to each individual member without having to store each of them. Then, the stack could store a lot more items, and tasks that currently require recursion (like pickling using the standard library) will be able to handle much deeper recursions. What do you think? Ram.

2 1

Python vs pypy: interesting performance difference [dict.setdefault]
by David Naylor 10 Mar '14

10 Mar '14

Hi, I needed to create a cache of date and time objects and I wondered what was the best way to handle the cache. For comparison I put together the following test: <file="iforkey.py"> import datetime import random import timeit ranges = [datetime.datetime(2011,01, random.randint(1, 31)) for i in xrange(1000)] def ifdict(): cache = {} b = [] for i in ranges: key = i.day if key in cache: b.append(cache[key]) else: date = i.date() cache[key] = date b.append(date) def keydict(): cache = {} b = [] for i in ranges: key = i.day try: b.append(cache[key]) except KeyError: date = i.date() cache[key] = date b.append(date) def defaultdict(): cache = {} b= [] for i in ranges: b.append(cache.setdefault(i, i.date())) print "ifdict:", timeit.repeat("ifdict()", "from __main__ import ifdict", number=10000) print "keydict:", timeit.repeat("keydict()", "from __main__ import keydict", number=10000) print "defaultdict:", timeit.repeat("defaultdict()", "from __main__ import defaultdict", number=10000) </file> # python iforfile.py ifdict: [2.432887077331543, 2.4002890586853027, 2.397233009338379] keydict: [2.3483030796051025, 2.358638048171997, 2.314802885055542] defaultdict: [3.5384328365325928, 3.5329859256744385, 3.5728111267089844] # pypy iforfile.py (pypy 1.5) ifdict: [0.8129069805145264, 0.74648118019104, 0.7432689666748047] keydict: [0.5187451839447021, 0.4662129878997803, 0.4504108428955078] defaultdict: [37.98510789871216, 37.859113931655884, 37.92770600318909] Pypy displays significant slowdown in the defaultdict function, otherwise displays its usual speedup. To check what is the cause I replaced i.date() with i.day and found no major difference in times. It appears dict.setdefault (or it's interaction with jit) is causing a slow down. Regards

4 14

numpy fails trigonometry with complex numbers, what to do?
by Matti Picus 23 Dec '13

23 Dec '13

I am trying to complete complex numbers in numpypy. Progress is good, I picked up from previous work on the numpypy-complex2 branch. Complex numbers come with extensive tests, it seems all the corner cases are covered. In porting the tests to numpypy, I came across a problem: numpy returns different results than cmath. Some of the differences are due to the fact that numpy does not raise a ValueError for dividing by 0 or other silly input values, but other differences are inexplicable (note the sign of the imaginary part): >>> numpy.arccos(complex(0.,-0.)) (1.5707963267948966-0j) >>> cmath.acos(complex(0.,-0.)) (1.5707963267948966+0j) >>> or this one: >>> cmath.acos(complex(float('inf'),2.3)) -infj >>> numpy.arccos(complex(float('inf'),2.3)) (0.78539816339744828-inf*j) Should I ignore the inconsistencies, or fix the 700 out of 2300 test instance failures? What should pypy's numpypy do - be consistent with numpy or with cmath? cmath is easier and probably faster (no need to mangle results or input args), so I would prefer cmath to trying to understand the logic behind numpy. Matti 2280c2365844

4 4

Windows 7 x64 development
by Robert Grosse 10 Sep '13

10 Sep '13

Hi, I am interested in working on Windows 7 64bit support for PyPy. Does anyone know what the best way to get started with this is? Is there a list of work that needs to be done to support Windows anywhere?

6 9

pypy-pyarray branch
by Antonio Cuni 31 Aug '13

31 Aug '13

Hi Matti, I and Romain reviewed the pypy-pyarray branch: we think there are a couple of issues to be solved before it can be merged, we added some comments to your TODO list. Otherwise, the branch looks fine and useful :) ciao, Anto. Romain

1 0

Re: [pypy-dev] [pypy-commit] pypy default: Unroll list.count() when the list is virtual or very small and constant lenght
by Carl Friedrich Bolz 30 Aug '13

30 Aug '13

Hi Alex, please revert this change, it can lead to combinatorial explosion: it can give a bridge per pattern of where in the list the object is found. Cheers, Carl Friedrich On 29/08/13 23:47, alex_gaynor wrote: > Author: Alex Gaynor <alex.gaynor(a)gmail.com> > Branch: > Changeset: r66432:185512e0c4df > Date: 2013-08-29 15:47 -0700 > http://bitbucket.org/pypy/pypy/changeset/185512e0c4df/ > > Log: Unroll list.count() when the list is virtual or very small and > constant lenght > > diff --git a/pypy/module/pypyjit/test_pypy_c/test_containers.py b/pypy/module/pypyjit/test_pypy_c/test_containers.py > --- a/pypy/module/pypyjit/test_pypy_c/test_containers.py > +++ b/pypy/module/pypyjit/test_pypy_c/test_containers.py > @@ -241,3 +241,22 @@ > loop, = log.loops_by_filename(self.filepath) > ops = loop.ops_by_id('getitem', include_guard_not_invalidated=False) > assert log.opnames(ops) == [] > + > + def test_list_count_virtual_list(self): > + def main(n): > + i = 0 > + while i < n: > + i += [n].count(n) > + return i > + > + log = self.run(main, [1000]) > + assert log.result == main(1000) > + loop, = log.loops_by_filename(self.filepath) > + assert loop.match(""" > + i7 = int_lt(i5, i6) > + guard_true(i7, descr=...) > + guard_not_invalidated(descr=...) > + i9 = int_add(i5, 1) > + --TICK-- > + jump(..., descr=...) > + """) > diff --git a/pypy/objspace/std/listobject.py b/pypy/objspace/std/listobject.py > --- a/pypy/objspace/std/listobject.py > +++ b/pypy/objspace/std/listobject.py > @@ -562,6 +562,8 @@ > 'L.reverse() -- reverse *IN PLACE*' > self.reverse() > > + @jit.look_inside_iff(lambda self, space, w_value: > + jit.loop_unrolling_heuristic(self, self.length(), UNROLL_CUTOFF)) > def descr_count(self, space, w_value): > '''L.count(value) -> integer -- return number of > occurrences of value''' > _______________________________________________ > pypy-commit mailing list > pypy-commit(a)python.org > http://mail.python.org/mailman/listinfo/pypy-commit >

3 5

Some benchmarking numbers
by Laurence Tratt 28 Aug '13

28 Aug '13

Some of you may remember an experiment that Carl Friedrich and I did at the beginning of this year, comparing PyPy against various other VMs. I've recently added V8 to the experiment, Tim Felgentreff has added Topaz, and I've updated all other VMs to the latest public release. See the numbers here: http://tratt.net/laurie/research/pubs/files/metatracing_vms/results/0.4/lin… All the normal cautions about over-interpreting benchmark results (particularly on synthetic micro-benchmarks) apply. Nevertheless, you may still find this interesting. In the future, it might be interesting to add the DeltaBlue benchmark for as many languages as we can find. I'm not going to put too much work into this, so patches are welcome: https://github.com/ltratt/metatracing_vms_experiment Laurie -- Personal http://tratt.net/laurie/ Software Development Team http://soft-dev.org/ https://github.com/ltratt http://twitter.com/laurencetratt

1 0

Quindi farvi fiducia? (Donazione) RICHIAMO
by Jacqueline Follereau 27 Aug '13

27 Aug '13

Ciao, Nomino Jacqueline Follereau, li avevo indirizzare un messaggio per spiegarvi la mia volontà, a seguito di ciò, il mio computer è guastarsi e non arrivo ad avere accesso al mio indirizzo posta elettronica. Per ciò vi chiedo di inviarmi la vostra risposta al mio nuovo indirizzo sul quale vi scrivo attualmente affinché possa rispondervi affinché possa darvi le istruzioni. Vi presento tutte le mie scuse per il fastidio involontario che è stato crea. Vi lascio sotto il messaggio che voi avevo lasciare: Ciao, Con molto rispetto per voi, con molto un buono cuore e la necessità di stabilire una comunicazione, è di conseguenza che vi scrivo. Io voi presi non considerato ciò poiché una scherza o un messaggio che vi causerà problemi. Volete considerare questo messaggio che ad estate inviarvi come una persona che cerca soprattutto la pace e la riservatezza del vostro aiuto. Il mio nome è la Signora JACQUELINE FOLLEREAU nato il 31-mag-1948, vedova senza figli. Sto contattando in questo modo perché voglio comunicare con il mondo e fare qualcosa che avrei forse non fatto, ma penso che dovremmo rischiare, confidando qualcuno. Sono affetto da cancro al cervello in malati terminali, e le mie giornate non sono più su un filo secondo l'ultimo analizza lo scanner che il mio medico mi ha detto per giustificare me stesso la palla che si deposita nel mio cervello per oltre 4 anni. Sono sterile vale a dire senza figli, ho intenzione di donare parte della mia proprietà. Ho lavorato nel business del petrolio per due anni presso la Refinery Company ivoriano (SIR), voglio un podi quel denaro va a varie associazioni, orfanotrofi e alle persone bisognose. Non importa chi siete, desidero con tutto il cuore di donare questo denaro del valore di circa euro 2.903.000 (Due Milioni Di Novecento e tre Mila euro) che viene aggiornato una società di sicurezza. Soffro tanto che faccio fatica a dormire, io sono turbato sapere che cosa sarebbe il denaro investito dopo la mia morte, non mi importa se si finisce nelle mani di qualcuno che è cosciente e timore di Dio con un cuore aperto e generoso può fare buon uso; Ma chi? Non lo so! Se è possibile con tutto il cuore e accettare questo compito, per favore fatemelo sapere, per favore! Vi prego di contattarmi al più presto se siete d'accordo a rispettare le mie istruzioni. Possa la pace e la misericordia di Dio sia con voi. Signora JACQUELINE FOLLEREAU

1 0

question about install pymongo
by Heng Zhou 25 Aug '13

25 Aug '13

I have installed the PyPy3 2.1 Beta 1 on my ubuntu. When I tried to import pymongo, a third party driver for mongodb, I was told "ImportError: No module named pymongo". So my questions are: (1)Does it mean I have to install pymongo again for pypy3 2.1 Beta 1? (2)If yes, http://doc.pypy.org/en/latest/getting-started.html#installing-pypy says: If you want to install 3rd party libraries, the most convenient way is to install distribute and pip: $ curl -O http://python-distribute.org/distribute_setup.py $ curl -O https://raw.github.com/pypa/pip/master/contrib/get-pip.py $ ./pypy-2.1/bin/pypy distribute_setup.py $ ./pypy-2.1/bin/pypy get-pip.py $ ./pypy-2.1/bin/pip install pygments # for example But, These instructions seems for pypy-2.1, the stable release for python 2.x only, not for PyPy3 which is used for python 3. Are the steps for installing 3rd party libraries for PyPy3 similar to the above ones? (3)If yes, I don't have curl installed in my ubuntu, do I have to install it? (4)I used "pip-3.3" which is particularly designed for python3 to install the pymongo instead of "pip". Is there any corresponding counterpart of pip-3.3 for PyPy3 2.1 Beta 1? If there is any, how to get it? Thank you.

2 2

Difficult in installing PyPy3 2.1 Beta 1 on ubuntu
by Heng Zhou 24 Aug '13

24 Aug '13

I downloaded the PyPy3 2.1 Beta 1 prebuilt binary file " https://bitbucket.org/pypy/pypy/downloads/pypy-2.1-linux64.tar.bz2" and unpack it. But When I run pypy in the bin directory, I was given an error: "The program 'pypy' is currently not installed. You can install it by typing: ..." instead of the success message in http://doc.pypy.org/en/latest/getting-started.html#id14. Actually this webpage says "PyPy is ready to be executed as soon as you unpack the tarball or the zip file, with no need to install it in any specific location:" Is there any particular install instruction for PyPy3 2.1 Beta 1 on Ubuntu as opposed to the web? Do I have to create a symlink? Btw, I'm using Ubuntu 13.04, is this mismatch of Ubuntu version the reason for the installation failure? Thank you.

2 1

pypy-dev August 2013