On Tue, Jul 20, 2021, at 14:41, Riadh wrote:
I found this presentation from Hinsen https://calcul.math.cnrs.fr/attachments/spip/IMG/pdf/cours_reproductibilite.pdf , correct me if I am wrong, but  I found nothing in contradiction with the proposed SKIP, please see 3rd bullet in slide 14. My understanding of Hinsen rule is  less forbiding  API breaks then saving dependencies version numbers...

Let's make the distinction clear:

An API break is when you do something like this:

v0: foo(x, rescale=True) -> out
v1: foo(x, rescale=True) -> error, rescale is not a valid keyword argument

The "Hinsen rule" (which is just a handle on the following concept; we could just as well call it "silent behavioral changes") is:

v0: foo(x) == y
v1: foo(x) != y

This last behavior change is a big problem, and one that should be avoided at all costs.  The prior is not so serious, because users get feedback when their code fails.  Once they fix it up, it works again and works correctly.

If you do not warn your users when behavior changes, then their code can deliver the wrong results without them knowing---and this is what we need to avoid.


P.S. With our existing deprecation mechanism, it is possible to hit a point in the future where we accidentally have the same API calls return different results. We should be cognizant of that failure mode, and avoid it.  One way is to never modify the "expected" output part of tests without careful consideration.