> Does it still make sense to add programs like this to our benchmarks?I agree that you should include them, I disagree that they are
> So far, our benchmarks are "real-life" examples. The benchmarks like
> above are completely missing the point of PyPy, as they don't stress
> at all the Python interpreter part. There are also other cases where
> PyPy's performance is very bad, like cpyext on an extension module
> with lots of small C API calls. I believe that it would still make
> sense to list such cases in the official benchmark, and have the
> descriptions of the benchmarks explain what's wrong with them.
'wrong'. They measure the overhead of a C call. Why should a C call
be slower in pypy than cpython? Presumably it could be compiled down
to the appropriate instructions and then out-perform cpy.