repeat itself is not slow, it's just that when it's used it iterates over it, in RPython (meaning it's not jit'd) which results in a dictionary lookup for the next() method at every iteration, which is slowish, list hits a special case so it doesn' thave that overhead. Alex On Thu, Jul 7, 2011 at 5:30 PM, Romain Guillebert <romain.py@gmail.com>wrote:

Hi

When I change this line:

primes[i*i:N+1:i] = repeat(False, len(primes[i*i:N+1:i]))

into this :

primes[i*i:N+1:i] = [False] * len(primes[i*i:N+1:i])

PyPy is much faster (but is still slower than CPython), so I would guess that the repeat function is the one to blame.

Cheers Romain

...

Hi.

I'm new to PyPy and was trying to run some tests to see orders of speed improvement.

Short script generating list of prime numbers using rather straightforward implementation of Eratosthene's sieve. Script: http://paste.pocoo.org/show/432727/

Typical results: http://paste.pocoo.org/show/432733/ (I thought that is due to absense of SSE2 on first computer, but I've rechecked on Intel(R) Xeon(R) CPU L5520 @ 2.27GHz with similar results).

I'm getting that CPython is nearly 4-8 times faster than PyPy. Is it a bug in PyPy or what is wrong (may be "specific" to PyPy) in my

On Fri, Jul 08, 2011 at 02:03:20AM +0300, Alexander Petrov wrote: script?

...

Alex. _______________________________________________ pypy-dev mailing list pypy-dev@python.org http://mail.python.org/mailman/listinfo/pypy-dev

_______________________________________________ pypy-dev mailing list pypy-dev@python.org http://mail.python.org/mailman/listinfo/pypy-dev

-- "I disapprove of what you say, but I will defend to the death your right to say it." -- Evelyn Beatrice Hall (summarizing Voltaire) "The people's good is the highest law." -- Cicero

No, you would need to implement list.__setitem__ in Python, which we could do, does the JIT see such code? Alex On Thu, Jul 7, 2011 at 6:05 PM, <exarkun@twistedmatrix.com> wrote:

On 12:38 am, alex.gaynor@gmail.com wrote:

...

repeat itself is not slow, it's just that when it's used it iterates over it, in RPython (meaning it's not jit'd) which results in a dictionary lookup for the next() method at every iteration, which is slowish, list hits a special case so it doesn' thave that overhead.

Is it time to reimplement repeat in Python then?

Jean-Paul

-- "I disapprove of what you say, but I will defend to the death your right to say it." -- Evelyn Beatrice Hall (summarizing Voltaire) "The people's good is the highest law." -- Cicero

Armin Rigo, 08.07.2011 11:18:

Before attacking the problem with the JIT, we should understand better why PyPy is 4-8 times slower than CPython. Normally you'd expect the factor to be at most 2. I suppose the answer is that our itertools.repeat() is bad for some reason.

You shouldn't forget that itertools contains extremely well optimised C code. I recently tried to (algorithmically) optimise the pure Python implementations from the CPython docs for Cython, and ended up with timings between 10% faster and 50% slower. It's actually hard to reach up to the raw power of hand tuned C code here. http://blog.behnel.de/index.php?p=163 http://blog.behnel.de/index.php?p=185 Given that some of the functions don't execute anything more than a couple of assembly instructions per cycle, you'll have to send a really well tuned high level implementation into the race to even come close. I would expect that PyPy's implementation simply wasn't written for that. That being said, it would be nice to have an itertools micro benchmark for the common Python test suite. Stefan

Thanks to all for the answers. Small remarks. I remember, I wrote such code (using itertools.repeat with slices) after spending near a day measuring performance of some syntactical constructs usually in several algorithms dealing with numbers... So I was trying to produce a sort of Python coding style for myself in that area - what constructs in what situations gave stable speed improvement and are stiil good-readable-pythonic. After that in all projects I used such things, e.g. using slice-with-repeat for assignment of constant, in prefer to for-loop or list-comprehension. Calculating primes by naive Eratosthene's sieve - is example of algorithm where a big part of calculation required to go beyong processor cache, so I suppose its performance is more characteristic of language-tools than very-low-level-compler-opmization or hardware architecture. Frankly speaking it was second piece of code that I have tested with PyPy - and suprised with such results :) Another one example I discovered where PyPy is slower is big numbers arithmetic (e.g. try to generate members of some recurrently defined sequence, such as Fibonacci, in a simple loop). =============== So at this time I didn't come to some kind of decision about PyPy. On one hand in most of the cases with straitforward code/algorithms and "common" syntax constructs there was significant speed improvement. But on the other hand, for the cases where source Python code was "optimized" or "hacked" the time of execution was sometimes better, sometimes of one order... and sometimes was a cause for this topic discussion. :) It is not bad thing generally, the bad thing that this speed degradation situations are happenned unexpectedly for me. IMO they are the most (and may be only one) interesting from the PyPy-user viewpoint. "Where and in what cases one can expect bottlenecks". Is there any documented collection of such artifacts? It can be exceptionally useful. Alex. On Sat, Jul 9, 2011 at 07:40, Alex Gaynor <alex.gaynor@gmail.com> wrote:

I'm not too sure what could be better wrong with it, it's rather short: https://bitbucket.org/pypy/pypy/src/default/pypy/module/itertools/interp_ite... Alex

On Fri, Jul 8, 2011 at 2:18 AM, Armin Rigo <arigo@tunes.org> wrote:

...

Hi Alex,

Before attacking the problem with the JIT, we should understand better why PyPy is 4-8 times slower than CPython.  Normally you'd expect the factor to be at most 2.  I suppose the answer is that our itertools.repeat() is bad for some reason.

A bientôt,

Armin.

-- "I disapprove of what you say, but I will defend to the death your right to say it." -- Evelyn Beatrice Hall (summarizing Voltaire) "The people's good is the highest law." -- Cicero

On Wed, Jul 20, 2011 at 9:25 AM, David Fraser <davidf@sjsoft.com> wrote:

On Wednesday, July 20, 2011, at 8:50:20 AM, "Alexander Petrov" <alexandervpetrov@gmail.com> wrote:

...

[snip] So at this time I didn't come to some kind of decision about PyPy.

On one hand in most of the cases with straitforward code/algorithms and "common" syntax constructs there was significant speed improvement.

But on the other hand, for the cases where source Python code was "optimized" or "hacked" the time of execution was sometimes better, sometimes of one order... and sometimes was a cause for this topic discussion. :) It is not bad thing generally, the bad thing that this speed degradation situations are happenned unexpectedly for me. IMO they are the most (and may be only one) interesting from the PyPy-user viewpoint. "Where and in what cases one can expect bottlenecks". Is there any documented collection of such artifacts? It can be exceptionally useful.

That's exactly what I would like. I also experimented with some simple tests and came out with PyPy being twice as slow as CPython - a wiki page which documents current areas of slowness, and potential workarounds would be fantastic - I know these things can be improved in the future, sometimes quickly, but it seems like the know-how about handling it in the mean time isn't written down anywhere...

David

https://bitbucket.org/pypy/pypy/wiki/JitFriendliness

On 07/20/2011 08:50 AM, Alexander Petrov wrote:

So at this time I didn't come to some kind of decision about PyPy.

On one hand in most of the cases with straitforward code/algorithms and "common" syntax constructs there was significant speed improvement.

But on the other hand, for the cases where source Python code was "optimized" or "hacked" the time of execution was sometimes better, sometimes of one order... and sometimes was a cause for this topic discussion. :) It is not bad thing generally, the bad thing that this speed degradation situations are happenned unexpectedly for me. IMO they are the most (and may be only one) interesting from the PyPy-user viewpoint. "Where and in what cases one can expect bottlenecks". Is there any documented collection of such artifacts? It can be exceptionally useful.

Another point is that PyPy considers these programs performance bugs to be fixed. So the situation you describe will likely change in the future. Of course some perform unpredictability will always remain, that goes with a JIT somewhat. Carl Friedrich