[Numpy-discussion] backwards compatibility and deprecation policy NEP

[Stephan Hoyer]

On Tue, Jul 24, 2018 at 5:38 PM Ralf Gommers <ralf.gommers at gmail.com> wrote:

> This is very developer-centric view. We have lots of users and also lots
> of no-longer-active contributors. The needs, interests and previous work
> put into NumPy of those groups of people matter.
>

Yes, I suppose it is :).

I tend to view NumPy's developers (interpreted somewhat broadly, including
those who contribute to the project in other ways) as the ultimate
representatives of NumPy's user base.


> I like would suggest the following criteria for considering removing a
>> NumPy submodule:
>>
> 1. It cannot be relied upon by other portions of NumPy.
>> 2. Either
>> (a) the submodule imposes a significant maintenance burden upon the rest
>> of NumPy that is not balanced by the level of dedicated contributions, or
>> (b) much better alternatives exist outside of NumPy
>>
>
> To quote Nathaniel: "the rest of our policy is all about measuring
> disruption based on effects on users". That's absent from your criteria.
>

Yes, "Can be achieved with minimum disruption for users" would be
appropriate to add as another top level criteria.

Why I would like to keep this point in is:
> - the discussion does come up, see draft brainstorm roadmap list and
> gh-11457.
> - the outcome of such discussions is in practice 100% clear.
> - I would like to avoid having drawn out discussions each time (this eats
> up a lot of energy for me), and I *really* would like to avoid saying "I
> don't have time to discuss, but this is just not going to happen" or
> "consider it vetoed".
> - Hence: just write it down, so we can refer to it.
>

I would rather we just say that the bar for deprecating or removing *any*
functionality in NumPy is extremely high. np.matrix is probably the best
example in recent times:
- np.matrix is officially discouraged (which we prefer even to deprecation)
- we *anticipate* deprecating it as soon as there's a viable alternative to
scipy.sparse
- even then, we will be very cautious about ever removing it, with the
understanding that it is widely used

As for updating this section of the NEP:
- We could certainly note that to date NumPy has not removed any complete
submodules (is this true?), and that these modules in particular, the
cost-benefit ratio does not favor removal at this time.
- Documenting the criteria we've come up with here, even though it hasn't
been satisfied yet, might be helpful to demonstrate the high bar that is
required.
- I don't like rejecting the possibility of removing submodules entirely
"simply not a good idea". It may become a good idea in the future, if some
of the underlying facts change.

I would also suggest highlighting two other strategies that NumPy uses in
favor of deprecation/removal:
- Official discouragement. Discouraging or deemphasizing in our docs is the
preferred strategy for older APIs that still have well defined behavior but
that are arguably less consistent with the rest of NumPy. Examples: isin vs
in1d, stack/block vs hstack/dstack/vstack.
- Benign neglect. This is our preferred strategy to removing submodules.
Merely being in NumPy does not automatically guarantee that a module is
well maintained, nor does it imply that a submodule is the best tool for
the job. That's OK, as long as the incremental maintenance burden on the
rest of NumPy is not too high.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/numpy-discussion/attachments/20180727/5b2ebbfd/attachment.html>