Hello, Out of curiosity, I timed running the test suite ("./python -m test.regrtest") in non-debug mode, in both the release30-maint and py3k branches: * release30-maint got: 302 tests OK. [...] 165.79user 26.03system 5:01.75elapsed 63%CPU * py3k got: 304 tests OK. [...] 113.33user 28.93system 4:06.79elapsed 57%CPU So, 3.1 is 30% faster in user CPU time, even though it probably has more tests (because of io-c, ordereddict and importlib). This is on a 64-bit Linux AMD system, and I got similar results on a 64-bit Linux Core2 system. Regards Antoine.
Antoine Pitrou wrote:
Hello,
Out of curiosity, I timed running the test suite ("./python -m test.regrtest") in non-debug mode, in both the release30-maint and py3k branches:
* release30-maint got: 302 tests OK. [...] 165.79user 26.03system 5:01.75elapsed 63%CPU
* py3k got: 304 tests OK. [...] 113.33user 28.93system 4:06.79elapsed 57%CPU
So, 3.1 is 30% faster in user CPU time, even though it probably has more tests (because of io-c, ordereddict and importlib). This is on a 64-bit Linux AMD system, and I got similar results on a 64-bit Linux Core2 system.
Regards
Antoine.
Yes, traceback in large file is also quite fast now. Good work, io-c guys. :-)
Hi,
So, 3.1 is 30% faster in user CPU time, even though it probably has more tests (...)
Wow! I just downloaded Python 2.6.1, 3.0.1 and 3.1alpha1, compiled them on 32 and 64 bits CPU, and ran pybench 2.1(*). Summary (minimum total) on 32 bits CPU: * Python 2.6.1: 8762 ms * Python 3.0.1: 8977 ms * Python 3.1a1: 9228 ms (slower than 3.0) Summary (minimum total) on 64 bits CPU: * Python 2.6.1: 4219 ms * Python 3.0.1: 4502 ms * Python 3.1a1: 4442 ms (faster than 3.0) I also ran pybench once, is it enough? See attached files for the details. (*) I copied pybench 2.1 from Pthon 3.1alpha1 because 2.6.1 only includes pybench 2.0 -- Victor Stinner aka haypo http://www.haypocalc.com/blog/
Antoine Pitrou wrote:
Hi,
Victor Stinner <victor.stinner <at> haypocalc.com> writes:
Summary (minimum total) on 32 bits CPU: * Python 2.6.1: 8762 ms * Python 3.0.1: 8977 ms * Python 3.1a1: 9228 ms (slower than 3.0)
Have you compiled with or without "--with-computed-gotos"?
Why is the feature still disabled by default? Christian PS: Holy moly! Computed gotos totally put my Python on fire! The feature increases the minimum run-time by approx. 25% and the average run-time by approx. 40% on my Ubuntu 8.10 box (AMD64, Intel(R) Core(TM)2 CPU T7600 @ 2.33GHz). ------------------------------------------------------------------------------- PYBENCH 2.1 ------------------------------------------------------------------------------- * using CPython 3.1a0 (py3k:70205, Mar 8 2009, 15:16:23) [GCC 4.3.2] * disabled garbage collection * system check interval set to maximum: 2147483647 * using timer: time.time ------------------------------------------------------------------------------- Benchmark: no_goto.pybench ------------------------------------------------------------------------------- Rounds: 10 Warp: 10 Timer: time.time Machine Details: Platform ID: Linux-2.6.27-11-generic-x86_64-with-debian-lenny-sid Processor: Python: Implementation: CPython Executable: /home/heimes/dev/python/py3k/python Version: 3.1.0 Compiler: GCC 4.3.2 Bits: 64bit Build: Mar 8 2009 15:11:16 (#py3k:70205) Unicode: UCS2 ------------------------------------------------------------------------------- Comparing with: goto.pybench ------------------------------------------------------------------------------- Rounds: 10 Warp: 10 Timer: time.time Machine Details: Platform ID: Linux-2.6.27-11-generic-x86_64-with-debian-lenny-sid Processor: Python: Implementation: CPython Executable: /home/heimes/dev/python/py3k/python Version: 3.1.0 Compiler: GCC 4.3.2 Bits: 64bit Build: Mar 8 2009 15:16:23 (#py3k:70205) Unicode: UCS2 Test minimum run-time average run-time this other diff this other diff ------------------------------------------------------------------------------- BuiltinFunctionCalls: 90ms 68ms +31.2% 109ms 75ms +44.8% BuiltinMethodLookup: 56ms 46ms +21.4% 75ms 50ms +48.9% CompareFloats: 100ms 61ms +64.8% 116ms 65ms +77.8% CompareFloatsIntegers: 139ms 97ms +43.3% 169ms 103ms +64.0% CompareIntegers: 151ms 88ms +72.7% 187ms 96ms +94.2% CompareInternedStrings: 117ms 66ms +78.9% 136ms 69ms +95.8% CompareLongs: 87ms 50ms +75.5% 102ms 53ms +93.1% CompareStrings: 102ms 53ms +94.0% 124ms 63ms +97.8% ComplexPythonFunctionCalls: 97ms 83ms +17.4% 113ms 85ms +32.9% ConcatStrings: 128ms 116ms +10.3% 151ms 117ms +28.6% CreateInstances: 90ms 81ms +10.0% 108ms 83ms +30.1% CreateNewInstances: 67ms 61ms +9.6% 84ms 63ms +32.7% CreateStringsWithConcat: 128ms 108ms +18.3% 150ms 110ms +35.6% DictCreation: 71ms 54ms +30.4% 87ms 61ms +43.3% DictWithFloatKeys: 83ms 67ms +24.1% 104ms 69ms +51.5% DictWithIntegerKeys: 80ms 60ms +32.9% 108ms 67ms +61.6% DictWithStringKeys: 109ms 50ms +117.0% 111ms 59ms +88.1% ForLoops: 70ms 59ms +18.4% 95ms 63ms +51.2% IfThenElse: 96ms 68ms +40.7% 115ms 69ms +66.8% ListSlicing: 78ms 77ms +1.5% 83ms 80ms +2.8% NestedForLoops: 92ms 71ms +29.2% 124ms 76ms +62.7% NestedListComprehensions: 98ms 76ms +28.6% 118ms 86ms +36.1% NormalClassAttribute: 152ms 139ms +9.2% 183ms 140ms +30.6% NormalInstanceAttribute: 96ms 76ms +27.2% 128ms 82ms +56.9% PythonFunctionCalls: 87ms 70ms +23.9% 100ms 71ms +39.9% PythonMethodCalls: 108ms 87ms +23.9% 126ms 91ms +38.9% Recursion: 135ms 113ms +20.0% 166ms 117ms +42.6% SecondImport: 84ms 82ms +3.0% 97ms 86ms +12.9% SecondPackageImport: 90ms 89ms +1.2% 102ms 91ms +12.0% SecondSubmoduleImport: 120ms 119ms +0.7% 130ms 120ms +8.4% SimpleComplexArithmetic: 68ms 57ms +18.3% 84ms 58ms +44.6% SimpleDictManipulation: 136ms 111ms +22.2% 162ms 115ms +41.2% SimpleFloatArithmetic: 83ms 53ms +56.7% 102ms 66ms +54.6% SimpleIntFloatArithmetic: 96ms 72ms +32.8% 119ms 79ms +51.0% SimpleIntegerArithmetic: 96ms 73ms +31.4% 121ms 73ms +65.5% SimpleListComprehensions: 82ms 68ms +20.6% 106ms 70ms +50.9% SimpleListManipulation: 77ms 52ms +47.7% 101ms 66ms +52.9% SimpleLongArithmetic: 66ms 55ms +19.8% 74ms 57ms +30.7% SmallLists: 101ms 81ms +25.8% 125ms 82ms +51.9% SmallTuples: 109ms 80ms +35.9% 137ms 86ms +59.5% SpecialClassAttribute: 252ms 249ms +1.2% 302ms 252ms +19.8% SpecialInstanceAttribute: 94ms 76ms +24.4% 117ms 86ms +35.4% StringMappings: 212ms 206ms +2.6% 220ms 213ms +3.6% StringPredicates: 90ms 79ms +14.6% 123ms 83ms +48.5% StringSlicing: 162ms 128ms +25.8% 175ms 138ms +27.4% TryExcept: 87ms 39ms +122.7% 108ms 47ms +128.0% TryFinally: 63ms 59ms +6.8% 80ms 68ms +17.9% TryRaiseExcept: 34ms 30ms +15.7% 47ms 30ms +57.5% TupleSlicing: 120ms 116ms +3.3% 133ms 118ms +12.2% WithFinally: 91ms 83ms +9.9% 123ms 91ms +34.8% WithRaiseExcept: 95ms 85ms +11.1% 114ms 86ms +32.0% ------------------------------------------------------------------------------- Totals: 5212ms 4185ms +24.5% 6274ms 4426ms +41.8% (this=no_goto.pybench, other=goto.pybench)
Christian Heimes <lists <at> cheimes.de> writes:
Why is the feature still disabled by default?
Marc-André expressed concerns that it might trigger compiler issues.
PS: Holy moly! Computed gotos totally put my Python on fire! The feature increases the minimum run-time by approx. 25% and the average run-time by approx. 40% on my Ubuntu 8.10 box (AMD64, Intel(R) Core(TM)2 CPU T7600 @ 2.33GHz).
You surely mean that it /decreased/ run-time by approx. 20% :-) I don't think average numbers are significant, but they might indicate that your system was slightly loaded when running the benchmarks. Regards Antoine.
On Sun, Mar 8, 2009 at 7:30 AM, Christian Heimes <lists@cheimes.de> wrote:
Antoine Pitrou wrote:
Hi,
Victor Stinner <victor.stinner <at> haypocalc.com> writes:
Summary (minimum total) on 32 bits CPU: * Python 2.6.1: 8762 ms * Python 3.0.1: 8977 ms * Python 3.1a1: 9228 ms (slower than 3.0)
Have you compiled with or without "--with-computed-gotos"?
Why is the feature still disabled by default?
Christian
PS: Holy moly! Computed gotos totally put my Python on fire! The feature increases the minimum run-time by approx. 25% and the average run-time by approx. 40% on my Ubuntu 8.10 box (AMD64, Intel(R) Core(TM)2 CPU T7600 @ 2.33GHz).
Note that of the benchmarks tested, PyBench benefits the most from threaded eval loop designs. Other systems benefit less; for example, Django template benchmarks were only sped up by 7-8% when I was testing it. Collin Winter
Le Sunday 08 March 2009 13:20:34 Antoine Pitrou, vous avez écrit :
Hi,
Victor Stinner <victor.stinner <at> haypocalc.com> writes:
Summary (minimum total) on 32 bits CPU: * Python 2.6.1: 8762 ms * Python 3.0.1: 8977 ms * Python 3.1a1: 9228 ms (slower than 3.0)
Have you compiled with or without "--with-computed-gotos"?
I used "./configure --prefix=...", no other option. Should I enable it? For which version? -- Victor Stinner aka haypo http://www.haypocalc.com/blog/
On Sun, Mar 8, 2009 at 4:05 AM, Victor Stinner <victor.stinner@haypocalc.com> wrote:
I just downloaded Python 2.6.1, 3.0.1 and 3.1alpha1, compiled them on 32 and 64 bits CPU, and ran pybench 2.1(*).
Summary (minimum total) on 32 bits CPU: * Python 2.6.1: 8762 ms * Python 3.0.1: 8977 ms * Python 3.1a1: 9228 ms (slower than 3.0)
Summary (minimum total) on 64 bits CPU: * Python 2.6.1: 4219 ms * Python 3.0.1: 4502 ms * Python 3.1a1: 4442 ms (faster than 3.0)
I also ran pybench once, is it enough? See attached files for the details.
(*) I copied pybench 2.1 from Pthon 3.1alpha1 because 2.6.1 only includes pybench 2.0
How are these numbers significant? IIUC that is not at all how pybench is supposed to be used. Its strength is that it lets you watch the relative performance of many individual operations. I don't think adding up the numbers for all operations gives a very useful total, since each individual timing loop seems to be scaled to last around 50-100 msec; this means the operation mix is probably vastly different from that occurring in real operations. What I'd be interested in however would be a list of which operations got speeded up the most and which slowed down the most. That might stir up someone's memory of a change that was made in that operations that could explain the performance change (especially for slow-downs, of course :-). -- --Guido van Rossum (home page: http://www.python.org/~guido/)
participants (6)
-
Antoine Pitrou -
Christian Heimes -
Collin Winter -
Guido van Rossum -
Hirokazu Yamamoto -
Victor Stinner