[Numpy-discussion] PEP 574 - zero-copy pickling with out of band data

[Andrea Gavana]

On Tue, 3 Jul 2018 at 07.35, Gael Varoquaux <gael.varoquaux at normalesup.org>
wrote:

> On Mon, Jul 02, 2018 at 05:31:05PM -0600, Charles R Harris wrote:
> > ISTR that some parallel processing applications sent pickled arrays
> around to
> > different processes, I don't know if that is still the case, but if so,
> no copy
> > might be a big gain for them.
>
> Yes, most parallel code that's across processes or across computers use
> some form a pickle. I hope that this PEP would enable large speed ups.
> This would be a big deal for parallelism in numerical Python.



This sound so very powerful... it’s such a pity that these type of gems
won’t be backported to Python 2 - we have so many legacy applications
smoothly running in Python 2 and nowhere near the required resources to
even start porting to Python 3, and pickle5 looks like  a small revolution
in the data-persistent world.

Andrea.



> _______________________________________________
> NumPy-Discussion mailing list
> NumPy-Discussion at python.org
> https://mail.python.org/mailman/listinfo/numpy-discussion
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/numpy-discussion/attachments/20180703/1e995d1b/attachment.html>