[Python-Dev] Benchmarks: Comparison between Python 3.5 and Python 3.6 performance

[Victor Stinner]

Hi,

Antoine Pitrou asked me to compare Python 3.6 to Python 3.5. Here are
results on the speed-python server using LTO compilation (but not
PGO):

$ python3 -m perf compare_to 2016-11-03_15-37-3.5-89f7386104e2.json
2016-11-03_15-38-3.6-c4319c0d0131.json -G --min-speed=5
Slower (17):
- call_method_slots: 16.7 ms +- 0.2 ms -> 28.3 ms +- 0.7 ms: 1.70x slower
- call_method: 16.9 ms +- 0.2 ms -> 28.6 ms +- 0.8 ms: 1.69x slower
- call_method_unknown: 18.5 ms +- 0.2 ms -> 30.8 ms +- 0.8 ms: 1.67x slower
- python_startup: 20.3 ms +- 0.7 ms -> 26.9 ms +- 0.6 ms: 1.33x slower
- regex_compile: 397 ms +- 4 ms -> 482 ms +- 6 ms: 1.21x slower
- mako: 42.2 ms +- 0.5 ms -> 49.7 ms +- 2.5 ms: 1.18x slower
- deltablue: 17.8 ms +- 0.2 ms -> 20.1 ms +- 0.6 ms: 1.13x slower
- chameleon: 29.1 ms +- 0.4 ms -> 31.9 ms +- 0.4 ms: 1.10x slower
- genshi_text: 89.8 ms +- 4.2 ms -> 97.8 ms +- 1.1 ms: 1.09x slower
- pickle_pure_python: 1.30 ms +- 0.02 ms -> 1.41 ms +- 0.03 ms: 1.08x slower
- logging_simple: 35.8 us +- 0.7 us -> 38.4 us +- 0.7 us: 1.07x slower
- sqlalchemy_declarative: 331 ms +- 3 ms -> 354 ms +- 6 ms: 1.07x slower
- logging_format: 42.6 us +- 0.5 us -> 45.5 us +- 0.8 us: 1.07x slower
- call_simple: 13.1 ms +- 0.4 ms -> 13.9 ms +- 0.2 ms: 1.06x slower
- genshi_xml: 197 ms +- 2 ms -> 209 ms +- 2 ms: 1.06x slower
- richards: 190 ms +- 5 ms -> 201 ms +- 6 ms: 1.06x slower
- go: 607 ms +- 19 ms -> 640 ms +- 26 ms: 1.05x slower

Faster (15):
- xml_etree_iterparse: 498 ms +- 11 ms -> 230 ms +- 5 ms: 2.17x faster
- unpickle_list: 10.6 us +- 0.3 us -> 7.86 us +- 0.16 us: 1.35x faster
- xml_etree_parse: 345 ms +- 6 ms -> 298 ms +- 8 ms: 1.16x faster
- xml_etree_generate: 362 ms +- 5 ms -> 320 ms +- 8 ms: 1.13x faster
- scimark_lu: 589 ms +- 24 ms -> 523 ms +- 18 ms: 1.13x faster
- regex_effbot: 5.88 ms +- 0.06 ms -> 5.23 ms +- 0.05 ms: 1.13x faster
- sympy_sum: 269 ms +- 7 ms -> 244 ms +- 7 ms: 1.10x faster
- spectral_norm: 317 ms +- 6 ms -> 287 ms +- 2 ms: 1.10x faster
- sympy_expand: 1.25 sec +- 0.04 sec -> 1.15 sec +- 0.03 sec: 1.09x faster
- xml_etree_process: 291 ms +- 5 ms -> 268 ms +- 14 ms: 1.09x faster
- float: 336 ms +- 8 ms -> 310 ms +- 7 ms: 1.08x faster
- regex_dna: 323 ms +- 3 ms -> 298 ms +- 3 ms: 1.08x faster
- unpickle: 38.4 us +- 0.6 us -> 35.5 us +- 1.2 us: 1.08x faster
- crypto_pyaes: 267 ms +- 3 ms -> 249 ms +- 2 ms: 1.07x faster
- nbody: 262 ms +- 3 ms -> 246 ms +- 3 ms: 1.07x faster

Benchmark hidden because not significant (31): 2to3, chaos, (...)
---

call_method_* slowdown: http://bugs.python.org/issue28618

python_startup slowdown: http://bugs.python.org/issue28637

I didn't analyze results yet.

Victor