[New-bugs-announce] [issue17787] Optimize pickling function lookups in hot loops.

[Alexandre Vassalotti]

New submission from Alexandre Vassalotti:

I found through profiling that batch_list_exact and batch_dict_exact spent a fair amount of time looking up the pickling function. I have found a way to optimize the lookup using a simple heuristic that assume items in dicts and lists have often a common type. So we have save some time by jumping right to the pickling function we need. Here are the result of running the benchmarks:

### fastpickle ###
Min: 0.593904 -> 0.541918: 1.10x faster
Avg: 0.606811 -> 0.564606: 1.07x faster
Significant (t=10.18)
Stddev: 0.01061 -> 0.02733: 2.5760x larger
Timeline: http://tinyurl.com/bwbvw7j

### pickle_list ###
Min: 0.907891 -> 0.915905: 1.01x slower
Avg: 0.970537 -> 0.948349: 1.02x faster
Significant (t=2.08)
Stddev: 0.07107 -> 0.02526: 2.8136x smaller
Timeline: http://tinyurl.com/cmlymq7

----------
assignee: alexandre.vassalotti
files: optimize_save_dispatch.patch
keywords: patch
messages: 187230
nosy: alexandre.vassalotti
priority: normal
severity: normal
stage: patch review
status: open
title: Optimize pickling function lookups in hot loops.
type: performance
versions: Python 3.4, Python 3.5
Added file: http://bugs.python.org/file29918/optimize_save_dispatch.patch

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue17787>
_______________________________________