[Numpy-discussion] PEP 574 - zero-copy pickling with out of band data

[Gael Varoquaux]

On Tue, Jul 03, 2018 at 08:54:51AM +0200, Andrea Gavana wrote:
> This sound so very powerful... it’s such a pity that these type of gems won’t
> be backported to Python 2 - we have so many legacy applications smoothly
> running in Python 2 and nowhere near the required resources to even start
> porting to Python 3,

I am a strong defender of stability and long-term support in scientific
software. But what you are demanding is that developers who do free work
do not benefit from their own work to have a more powerful environment.

More recent versions of Python are improved compared to older ones and
make it much easier to write certain idioms. Developers make these
changes over years to ensure that codebases are always simpler and more
robust. Backporting in effect means doing this work twice, but the second
time with more constraints. I just allocated something like a man-year to
have robust parallel-computing features work both on Python 2 and Python
3. With this man-year we could have done many other things. Did I make
the correct decision? I am not sure, because this is just creating more
technical dept.

I understand that we all sit on piles of code that we wrote for a given
application and one point, and that we will not be able to modernise it
all. But the fact that we don't have the bandwidth to make it evolve
probably means that we need to triage what's important and call a loss
the rest. Just like if I have 5 old cars in my backyard, I won't be able
to keep them all on the road unless I am very rich.


People asking for infinite backport to Python 2 are just asking
developers to write them a second free check, even larger than the one
they just got by having the feature under Python 3.


Gaël