
First: benchmarks (especially those which are micro) sucks. It seems that these days we're vastly faster on microbenchs than cpython 2.5 and way slower on some tests from pybench (like small tuple operations). Here are some results (it's the other way around than on tuatara, higher is better, more than 1 means faster than cpython). This only showcases how much microbenchmarks are dependand on tight loop speed in my opinion :-( Pybench is slightly smarter in this regard. 0.22x slower on test_unicode.test_bad_case_python2_5() 0.26x slower on test_formatting.test_format_unicode() 0.26x slower on test_formatting.test_format_long() 0.78x slower on test_exception.test_reraise() 0.83x slower on test_bltn.test_call_fabs() 0.84x slower on test_exception.test_except_specific_builtin_exception() 0.87x slower on test_exception.test_except_multiple_builtin_exception() 0.87x slower on test_bltn.test_call_sin() 0.91x slower on test_exception.test_except_specific_user_exception() 0.94x slower on test_count1.test_loop_unrolled() 0.94x slower on test_count1.test_loop() 0.97x slower on test_dict.test_dict_creation_mode3() 1.01x slower on test_exception.test_except_multiple_user_exception() 1.10x slower on test_bltn.test_isinstance2() 1.12x slower on test_count1.test_call_function_with_arguments_in_cellvars() 1.15x slower on test_count1.test_count_in_attr() 1.15x slower on test_dispatch.test_dispatch_nop() 1.18x slower on test_unicode.test_find_pattern1() 1.19x slower on test_bltn.test_isinstance1() 1.19x slower on test_count1.test_loop_other_count() 1.23x slower on test_exception.test_raise_builtin_exception() 1.23x slower on test_count1.test_call_function_without_arguments_in_cellvars() 1.24x slower on test_count1.test_call_method_of_old_style_class() 1.28x slower on test_count1.test_count_in_global() 1.29x slower on test_exception.test_raise_user_exception() 1.29x slower on test_count1.test_call_method_of_new_style_class() 1.29x slower on test_bltn.test_isinstance3() 1.30x slower on test_count1.test_count_in_global2() 1.30x slower on test_formatting.test_number_formatting() 1.32x slower on test_create1.test_simple_loop_with_new_style_class_new() 1.35x slower on test_exception.test_try_except_finally() 1.36x slower on test_count1.test_count_in_dict() 1.37x slower on test_exception.test_try_except_else() 1.38x slower on test_count1.test_call_nested_function_many_args() 1.39x slower on test_dict.test_dict_class_dict_getmethod() 1.39x slower on test_exception.test_try_except() 1.40x slower on test_count1.test_call_nested_function() 1.40x slower on test_create1.test_simple_loop() 1.42x slower on test_dict.test_dict_instance_setattr_instance_dict() 1.44x slower on test_dict.test_dict_instance_setnewattr_instance_dict() 1.45x slower on test_count1.test_call_nested_function_other_count() 1.49x slower on test_list.test_list_setitem() 1.55x slower on test_list.test_list_extend() 1.59x slower on test_count1.test_call_function() 1.63x slower on test_dict.test_dict_getitem() 1.64x slower on test_count1.test_count_in_slot() 1.65x slower on test_formatting.test_dict_formatting() 1.68x slower on test_dict.test_dict_creation_mode1() 1.72x slower on test_unicode.test_find_pattern2() 1.75x slower on test_dict.test_dict_setitem2() 1.75x slower on test_dict.test_dict_raw_range() 1.75x slower on test_dict.test_dict_creation_mode2() 1.76x slower on test_dict.test_dict_instance_getattr_instance_dict() 1.80x slower on test_list.test_list_append() 1.92x slower on test_create1.test_simple_loop_with_old_style_class_creation() 2.01x slower on test_formatting.test_simple_formatting() 2.12x slower on test_formatting.test_evil_formatting() 2.27x slower on test_create1.test_simple_loop_with_new_style_class_creation() 2.37x slower on test_list.test_list_getitem() 2.55x slower on test_dict.test_dict_setitem1() 2.59x slower on test_unicode.test_find_worstcase() 2.67x slower on test_unicode.test_find_pattern4() 2.95x slower on test_list.test_list_slice() 3.24x slower on test_unicode.test_count_worstcase() 3.27x slower on test_exception.test_instantiate_user_exception() 3.64x slower on test_exception.test_instantiate_builtin_exception() 3.73x slower on test_formatting.test_repr_formatting() 4.22x slower on test_unicode.test_find_pattern8() 6.32x slower on test_unicode.test_find_pattern16() PyPy-c compiled using --gc=generation --gcrootfinder=asmgcc --faassen cpython 2.5 from release On pybench we're 70% slower in general than cpython, 55% slower with --objspace-std-withstrslice, comparison table of withstrslice (positive numbers means slowdown compared to cpython, negative means speedup, in %): Tests: per run per oper. diff *) ------------------------------------------------------------------------ BuiltinFunctionCalls: 37.70 ms 0.30 us -7.82% BuiltinMethodLookup: 133.65 ms 0.25 us +47.27% CompareFloats: 71.55 ms 0.16 us +5.53% CompareFloatsIntegers: 376.70 ms 0.84 us +339.04% CompareIntegers: 69.20 ms 0.08 us -19.25% CompareInternedStrings: 91.60 ms 0.18 us +51.66% CompareLongs: 71.55 ms 0.16 us +19.65% CompareStrings: 104.65 ms 0.21 us +17.12% CompareUnicode: 67.95 ms 0.18 us -25.78% ConcatStrings: 28.30 ms 0.19 us -70.40% ConcatUnicode: 442.10 ms 2.95 us +225.67% CreateInstances: 81.60 ms 1.94 us +20.18% CreateStringsWithConcat: 46.40 ms 0.23 us +8.54% CreateUnicodeWithConcat: 107.45 ms 0.54 us +7.56% DictCreation: 104.65 ms 0.70 us +77.67% DictWithFloatKeys: 311.75 ms 0.52 us +135.46% DictWithIntegerKeys: 171.65 ms 0.29 us +113.89% DictWithStringKeys: 105.70 ms 0.18 us +33.04% ForLoops: 147.00 ms 14.70 us +145.00% IfThenElse: 79.15 ms 0.12 us +4.77% ListSlicing: 52.00 ms 14.86 us +63.27% NestedForLoops: 93.00 ms 0.27 us +126.83% NormalClassAttribute: 185.40 ms 0.31 us +114.09% NormalInstanceAttribute: 113.25 ms 0.19 us +40.25% PythonFunctionCalls: 99.55 ms 0.60 us +21.92% PythonMethodCalls: 82.75 ms 1.10 us +23.69% Recursion: 86.65 ms 6.93 us +46.99% SecondImport: 51.90 ms 2.08 us +44.17% SecondPackageImport: 34.95 ms 1.40 us -7.66% SecondSubmoduleImport: 77.75 ms 3.11 us +52.45% SimpleComplexArithmetic: 67.75 ms 0.31 us +47.60% SimpleDictManipulation: 105.70 ms 0.35 us +104.05% SimpleFloatArithmetic: 80.20 ms 0.15 us -4.18% SimpleIntFloatArithmetic: 87.15 ms 0.13 us +21.72% SimpleIntegerArithmetic: 89.20 ms 0.14 us +22.95% SimpleListManipulation: 54.50 ms 0.20 us +21.92% SimpleLongArithmetic: 63.80 ms 0.39 us +64.01% SmallLists: 66.30 ms 0.26 us -6.09% SmallTuples: 250.55 ms 1.04 us +270.36% SpecialClassAttribute: 197.45 ms 0.33 us +127.61% SpecialInstanceAttribute: 114.35 ms 0.19 us -14.41% StringMappings: 104.10 ms 0.83 us +30.86% StringPredicates: 68.25 ms 0.24 us -10.02% StringSlicing: 64.30 ms 0.37 us +17.87% TryExcept: 105.80 ms 0.07 us +15.82% TryRaiseExcept: 37.55 ms 2.50 us +14.31% TupleSlicing: 91.65 ms 0.87 us +43.54% UnicodeMappings: 132.60 ms 7.37 us +67.11% UnicodePredicates: 56.95 ms 0.25 us -15.75% UnicodeSlicing: 169.15 ms 0.97 us +123.60% ------------------------------------------------------------------------ Average round time: 6238.00 ms +55.21% Cheers, fijal

On 28/04/2008, at 11:38, Maciej Fijalkowski wrote:
One important question is, can we profile some real python programs and see the same behavior in slowdowns? Because right now what you are doing is comparing two synthetic benchmarks, which I think is not as usefull for pypython. The python interpreter needs better benchmarks... that is a given.

On 28/04/2008, at 11:38, Maciej Fijalkowski wrote:
One important question is, can we profile some real python programs and see the same behavior in slowdowns? Because right now what you are doing is comparing two synthetic benchmarks, which I think is not as usefull for pypython. The python interpreter needs better benchmarks... that is a given.
participants (2)
-
Leonardo Santagada
-
Maciej Fijalkowski