[Numpy-discussion] backwards compatibility and deprecation policy NEP

[Stephan Hoyer]

On Sat, Jul 21, 2018 at 6:40 PM Marten van Kerkwijk <
m.h.vankerkwijk at gmail.com> wrote:

> But I think the subclassing section is somewhat misleading in suggesting
> `ndarray` is not well designed to be subclassed. At least, for neither my
> work on Quantity nor that on MaskedArray, I've found that the design of
> `ndarray` itself was a problem. Instead, it was the functions that were, as
> most were not written with subclassing or duck typing in mind, but rather
> with the assumption that all input should be an array, and that somehow it
> is useful to pass anything users pass in through `asarray`. With then
> layers on top to avoid this in specific circumstances... But perhaps this
> is what you meant?
>

I can't speak for Ralf, but yes, this is part of what I had in mind. I
don't think you can separate "core" objects/methods from functions that act
on them. Either the entire system is designed to handle subclassing through
some well-defined interface or is it not.

If you don't design a system for subclassing but allow it anyways (and it's
impossible to prohibit problematically in Python), then you can easily end
up with very fragile systems that are difficult to modify or extend. As
Ralf noted in the NEP, "Some of them change the behavior of ndarray
methods, making it difficult to write code that accepts array duck-types."
These changes end up having implications for apparently unrelated functions
(e.g., np.median needing to call np.mean internally to handle units
properly). I don't think anyone really wants that sort of behavior or
lock-in in NumPy itself, but of course that is the price we pay for not
having well-defined interfaces :). Hopefully NEP-18 will change that, and
eventually we will be able to remove hacks from NumPy that we added only
because there weren't any better alternatives available.

For the NEP itself, i would not mention "A future change in NumPy to not
support subclassing," because it's not as if subclassing is suddenly not
going to work as of a certain NumPy release.  Certain types of subclasses
(e.g., those that only add extra methods and/or metadata and do not modify
any existing functionality) have never been a problem and will be fine to
support indefinitely.

Rather, we might state that "At some point in the future, the NumPy
development team may no longer interested in maintaining workarounds for
specific subclasses, because other interfaces for extending NumPy are
believed to be more maintainable/preferred."

Overall, it seems to me that these days in the python eco-system
> subclassing is simply expected to work.
>

I don't think this is true. You can use subclassing on builtin types like
dict, but just because you can do it doesn't mean it's a good idea. If you
change built-in methods to work in different ways other things will break
in unexpected ways (or simply not change, also in unexpected ways).
Probably the only really safe way to subclass a dictionary is to define the
__missing__() method and not change any other aspects of the public
interface directly.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/numpy-discussion/attachments/20180723/b3cfecc5/attachment.html>