Hello,
On Sat, 17 Nov 2012 23:18:29 +0200
Serhiy Storchaka
I have an idea which can increase pickle/unpickle performance. This requires a change of format, so we need a new version of the protocol. May be it will be a 4 (PEP 3154) or 5.
All items should be aligned to 4 bytes. It allow fast reading/writing of small integers, memcpy and UTF-8 codec should be faster on aligned data.
I can see several drawbacks here: - you will still have to support unaligned data (because of memoryview slices) - it may not be significantly faster, because actual creation of objects also contributes to unpickling performance - there will be less code sharing between different protocol versions, making pickle harder to maintain If you think this is worthwhile, I think you should first draft a proof of concept to evaluate the performance gain. Regards Antoine.