[Python-Dev] Cython for cPickle?

[Alexandre Vassalotti]

On Thu, Apr 19, 2012 at 4:55 AM, Stefan Behnel <stefan_ml at behnel.de> wrote:
>
> That sounds like less than two weeks of work, maybe even if we add the
> marshal module to it.
> In less than a month of GSoC time, this could easily reach a point where
> it's "close to the speed of what we have" and "fast enough", but a lot more
> accessible and maintainable, thus also making it easier to add the
> extensions described in the PEP.
>
> What do you think?


As others have pointed out, many users of pickle depend on its performance.
The main reason why _pickle.c is so big is all the low-level optimizations
we have in there. We have custom stack and dictionary implementations just
for the sake of speed. We also have fast paths for I/O operations and
function calls. These optimizations alone are taking easily 2000 lines of
code and they are not micro-optimizations. Each of these were shown to give
speedups from one to several orders of magnitude.

So I disagree that we could easily reach the point where it's "close to the
speed of what we have." And if we were to attempt this, it would be a
multiple months undertaking. I would rather see that time spent on
improving pickle than on yet another reimplementation.

-- Alexandre
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20120422/8b77abff/attachment.html>