Hi Victor,

On 2 November 2016 at 11:50, Victor Stinner victor.stinner@gmail.com wrote:

2016-11-02 11:04 GMT+01:00 Armin Rigo armin.rigo@gmail.com:

...

On 19 October 2016 at 18:55, Victor Stinner victor.stinner@gmail.com wrote:

...

3. new --duplication option to perf timeit

This is never a good idea on top of PyPy, so I wouldn't mind if using this option on top of PyPy threw an error.

Can you please elaborate?

Yes, exactly :-) Consider a benchmark written like that:

for i in range(lots):
     z = a + b
     z = a + b
     z = a + b
     z = a + b
     z = a + b

What you are really measuring by running PyPy on this is completely different from what you *think* you are measuring---in this case, mostly everything is optimized away. If you try to make it actually do something so that it's not optimized away, then the problem of duplicating lines becomes of making the tracing JIT compiler not happy at all. If you duplicate the lines too many times, the loop body becomes too long for the JIT compiler to swallow---never duplicates 1000 times, that's always too much! But even if you duplicate only 10 times, then the more subtle problem is: assume that each line can follow *two* control flow paths (even internally, e.g. because of some condition done in RPython). (It is likely the case, if you try to do something non-trivially-optimizable-away.) Then if you duplicate the line 10 times, there are suddenly 2**10 control flow paths. That means the JIT will never be able to warm up completely. Suddenly you are measuring the JIT compiler's performance and not at all your code's.

The --duplication option on PyPy is thus either useless or limited to use cases where you definitely know there is only one code path ever followed, and don't duplicate too much, and know for sure that multiple repetitions of the same line won't cause cross-line optimizations. That's not possible to explain without going very technical.

A bientôt,

Armin.