[Python-ideas] Fast pickle

[Antoine Pitrou]

Hello,

On Sat, 17 Nov 2012 23:18:29 +0200
Serhiy Storchaka <storchaka at gmail.com>
wrote:
> I have an idea which can increase pickle/unpickle performance. This requires a change of format, so we need a new version of the protocol. May be it will be a 4 (PEP 3154) or 5.
> 
> All items should be aligned to 4 bytes. It allow fast reading/writing of small integers, memcpy and UTF-8 codec should be faster on aligned data.

I can see several drawbacks here:
- you will still have to support unaligned data (because of memoryview
  slices)
- it may not be significantly faster, because actual creation of
  objects also contributes to unpickling performance
- there will be less code sharing between different protocol versions,
  making pickle harder to maintain

If you think this is worthwhile, I think you should first draft a
proof of concept to evaluate the performance gain.

Regards

Antoine.