Begin forwarded message:From: Stephan Hoyer
Date: Friday, Nov 09, 2018 at 3:19 PM
To: Hameer Abbasi
Cc: Stefan van der Walt, Marten van Kerkwijk
Subject: asarray/anyarray; matrix/subclassThis is a great discussion, but let's try to have it in public (e.g., on the NumPy mailing list).On Fri, Nov 9, 2018 at 8:42 AM Hameer Abbasi <einstein.edison@gmail.com> wrote:Hi Stephan,The issue I have with writing another function is that asarray/asanyarray are so widely used that it’d be a huge maintenance task to update them throughout NumPy, not to mention other codebases, not to mention other codebases having to rely on newer NumPy versions for this. In short, it would dramatically reduce adaptability of this function.One path we can take is to allow asarray/asanyarray to be overridable via __array_function__ (the former is debatable). This solves most of our duck-array related issues without introducing another protocol.Regardless of what path we choose, I would recommend changing asanyarray to not pass through np.matrix regardless, instead passing through mat.view(type=np.ndarray) instead, which has O(1) cost and memory. In the vast majority of contexts, it’s used to ensure an array-ish structure for another operation, and usually there’s no guarantee that what comes out will be a matrix anyway. I suggest we raise a FutureWarning and then change this behaviour.There have been a number of discussions about deprecating np.matrix (and a few about MaskedArray as well, though there are less compelling reasons for that one). I suggest we start down that path as soon as possible. The biggest (only?) user I know of blocking that is scipy.sparse, and we’re on our way to replacing that with PyData/Sparse.Best Regards,Hameer AbbasiOn Friday, Nov 09, 2018 at 1:26 AM, Stephan Hoyer <shoyer@gmail.com> wrote:Hi Hameer,
I'd love to talk about this in more detail. I agree that something like this is needed.
The challenge with reusing an existing function like asanyarray() is that there is at least one (somewhat?) widely used ndarray subclass that badly violates the Liskov Substitution Principle: np.matrix.
NumPy can't really use np.asanyarray() widely for internal purposes until we don't have to worry about np matrix. We might special case np.matrix in some way, but then asanyarray() would do totally opposite things on different versions of NumPy. It's almost certainly a better idea to just write a new function with the desired semantics, and "soft deprecate" asanyarray(). The new function can explicitly black list np.matrix, as well as any other subclasses we know of that badly violate LSP.
Cheers,
StephanOn Thu, Nov 8, 2018 at 5:06 PM Hameer Abbasi <einstein.edison@gmail.com> wrote:No, Stefan, I’ll do that now. Putting you in the cc.It slipped my mind among the million other things I had in mind — Namely: My job visa. It was only done this Monday.Hi, Marten, Stephan:Stefan wants me to write up a NEP that allows a given object to specify that it is a duck array — Namely, that it follows duck-array semantics.We were thinking of switching asanyarray to switch to passing through anything that implements the duck-array protocol along with ndarray subclasses. I’m sure this would help XArray and Quantity work better with existing codebases, along with PyData/Sparse arrays.Would you be interested?
Best Regards,Hameer AbbasiOn Thursday, Nov 08, 2018 at 9:09 PM, Stefan van der Walt <stefanv@berkeley.edu> wrote:Hi Hameer,
In last week's meeting, we had the following in the notes:
Hameer is contacting Marten & Stephan and write up a draft NEP for
clarifying the asarray/asanyarray and matrix/subclass path forward.
Did any of that happen that you could share?
Thanks and best regards,
Stéfan