-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 - From what I understand, neither pystone nor richards are very representative of idiomatic python, but pybench is (as much as is possible for a synthetic benchmark, in any case). So, I thought it would be interesting to see how pypy compared to cpython using pybench. Pybench 2.0 was used, downloaded from the python.org svn repo. Pybench reports details of the machine and python installation using the various functions in the platform module, but these functions don't appear to work in pypy, so the reporting was disabled. The UnicodeProperties test also had to be disabled due to pypy not having a unicodedata module. All other tests ran successfully, and pybench was otherwise unmodified. pypy revision 32952 was used, but pypy was translated using the default options, so I imagine these results could be improved on. The tests are run ten times, the first group of three columns shows the best times, and the second group shows the average times. Within each group, the first column is the pypy time, the second the python 2.4.3 time, and the third is the ratio. I hope these results are of interest, and if anyone has any advice on how to improve the test setup, or translation time options I should try, please let me know. Jonathan Doda Test minimum average - ------------------------------------------------------------------------ BuiltinFunctionCalls: 343ms 178ms 1.93 368ms 179ms 2.06 BuiltinMethodLookup: 554ms 161ms 3.44 567ms 174ms 3.26 CompareFloats: 285ms 172ms 1.66 295ms 173ms 1.71 CompareFloatsIntegers: 1492ms 171ms 8.73 1576ms 171ms 9.22 CompareIntegers: 410ms 160ms 2.56 425ms 160ms 2.66 CompareInternedStrings: 414ms 164ms 2.52 421ms 165ms 2.55 CompareLongs: 251ms 137ms 1.83 261ms 138ms 1.89 CompareStrings: 279ms 145ms 1.92 293ms 146ms 2.01 CompareUnicode: 315ms 145ms 2.17 320ms 146ms 2.19 ConcatStrings: 2230ms 201ms 11.09 2251ms 209ms 10.77 ConcatUnicode: 5515ms 209ms 26.39 5570ms 213ms 26.15 CreateInstances: 953ms 180ms 5.29 1000ms 189ms 5.29 CreateNewInstances: 712ms 165ms 4.32 746ms 166ms 4.49 CreateStringsWithConcat: 1067ms 175ms 6.10 1078ms 178ms 6.06 CreateUnicodeWithConcat: 905ms 167ms 5.42 919ms 175ms 5.25 DictCreation: 661ms 136ms 4.86 686ms 136ms 5.04 DictWithFloatKeys: 811ms 191ms 4.25 826ms 192ms 4.30 DictWithIntegerKeys: 482ms 154ms 3.13 493ms 155ms 3.18 DictWithStringKeys: 584ms 143ms 4.08 592ms 144ms 4.11 ForLoops: 898ms 150ms 5.99 905ms 152ms 5.95 IfThenElse: 388ms 135ms 2.87 396ms 136ms 2.91 ListSlicing: 423ms 106ms 3.99 454ms 107ms 4.24 NestedForLoops: 1137ms 176ms 6.46 1141ms 178ms 6.41 NormalClassAttribute: 559ms 156ms 3.58 567ms 157ms 3.61 NormalInstanceAttribute: 935ms 143ms 6.54 945ms 144ms 6.56 PythonFunctionCalls: 1048ms 133ms 7.88 1057ms 134ms 7.89 PythonMethodCalls: 1275ms 168ms 7.59 1285ms 169ms 7.60 Recursion: 1599ms 185ms 8.64 1607ms 186ms 8.64 SecondImport: 685ms 137ms 5.00 694ms 138ms 5.03 SecondPackageImport: 602ms 146ms 4.12 618ms 149ms 4.15 SecondSubmoduleImport: 987ms 179ms 5.51 996ms 182ms 5.47 SimpleComplexArithmetic: 745ms 197ms 3.78 750ms 201ms 3.73 SimpleDictManipulation: 668ms 151ms 4.42 670ms 153ms 4.38 SimpleFloatArithmetic: 396ms 172ms 2.30 407ms 182ms 2.24 SimpleIntFloatArithmetic: 363ms 131ms 2.77 376ms 133ms 2.83 SimpleIntegerArithmetic: 367ms 132ms 2.78 377ms 133ms 2.83 SimpleListManipulation: 384ms 121ms 3.17 399ms 123ms 3.24 SimpleLongArithmetic: 651ms 144ms 4.52 660ms 145ms 4.55 SmallLists: 371ms 140ms 2.65 372ms 142ms 2.62 SmallTuples: 2122ms 148ms 14.34 2144ms 150ms 14.29 SpecialClassAttribute: 545ms 154ms 3.54 573ms 155ms 3.70 SpecialInstanceAttribute: 939ms 182ms 5.16 971ms 184ms 5.28 StringMappings: 517ms 170ms 3.04 539ms 172ms 3.13 StringPredicates: 491ms 160ms 3.07 518ms 168ms 3.08 StringSlicing: 994ms 154ms 6.45 1039ms 160ms 6.49 TryExcept: 424ms 156ms 2.72 444ms 157ms 2.83 TryRaiseExcept: 666ms 182ms 3.66 687ms 184ms 3.73 TupleSlicing: 521ms 163ms 3.20 547ms 165ms 3.32 UnicodeMappings: 389ms 168ms 2.32 403ms 170ms 2.37 UnicodePredicates: 433ms 152ms 2.85 449ms 157ms 2.86 UnicodeSlicing: 1874ms 180ms 10.41 1922ms 186ms 10.33 Totals: 42658ms 8125ms 5.25 43595ms 8261ms 5.28 -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.2.2 (GNU/Linux) iD8DBQFFJoQbWEgeeLQ8a74RAh36AJ9J0MwiaaM6aqAnvQidrTCCeSyKMQCfWvRZ vyz4YzrY0oEYzaAgBd9Ny+0= =Qr6G -----END PGP SIGNATURE-----
Hi Jonathan! Thanks for doing this, we used pybench a while ago but didn't pursue if further. This is indeed very interesting! Jonathan Doda wrote:
- From what I understand, neither pystone nor richards are very representative of idiomatic python, but pybench is (as much as is possible for a synthetic benchmark, in any case). So, I thought it would be interesting to see how pypy compared to cpython using pybench.
Pybench 2.0 was used, downloaded from the python.org svn repo. Pybench reports details of the machine and python installation using the various functions in the platform module, but these functions don't appear to work in pypy, so the reporting was disabled. The UnicodeProperties test also had to be disabled due to pypy not having a unicodedata module. All other tests ran successfully, and pybench was otherwise unmodified.
PyPy has a unicodedata module, it is not compiled in by default, though. It's not clear to me why the platform module does not work, though.
pypy revision 32952 was used, but pypy was translated using the default options, so I imagine these results could be improved on.
The tests are run ten times, the first group of three columns shows the best times, and the second group shows the average times. Within each group, the first column is the pypy time, the second the python 2.4.3 time, and the third is the ratio.
I hope these results are of interest, and if anyone has any advice on how to improve the test setup, or translation time options I should try, please let me know.
You could try to pass in some translation-options that try to optimize common string operations and small integers: python translate.py targetpypystandalone.py --objspace-std-withsmallint --objspace-std-withstrslice --objspace-std-withstrjoin Other interesting options are --objspace-std-withstrdict (to enable optimizations for dictionaries with string keys), --usemodules=unicodedata (to enable the unicode data module). The options are mutually compatible, so you can do a super-build with all of them (which unfortunately increases executable size quite a bit). Cheers, Carl Friedrich Bolz
On Saturday 07 October 2006 04:15, Carl Friedrich Bolz wrote:
PyPy has a unicodedata module, it is not compiled in by default, though. It's not clear to me why the platform module does not work, though.
'platform.py' also doesn't work with IronPython. I posted a patch on SF for this - www.python.org/sf/1563842 There's another patch with pybench fixes for IronPython that might be related. Don't have the patch# right now. Making it work with pypy is probably worthwhile, too. Someone working on the project might want to add a note to the patch. -- Anthony Baxter <anthony@interlink.com.au> It's never too late to have a happy childhood.
participants (3)
-
Anthony Baxter
-
Carl Friedrich Bolz
-
Jonathan Doda