On Sun, Jun 2, 2019 at 1:08 PM Marten van Kerkwijk <m.h.vankerkwijk@gmail.com> wrote:


On Sun, Jun 2, 2019 at 2:21 PM Eric Wieser <wieser.eric+numpy@gmail.com> wrote:
Some of your categories here sound like they might be suitable for ABCs that provide mixin methods, which is something I think Hameer suggested in the past. Perhaps it's worth re-exploring that avenue.

Eric


Indeed, and of course for __array_ufunc__ we moved there a bit already, with `NDArrayOperatorsMixin` [1].
One could certainly similarly have NDShapingMixin that, e.g., relied on `shape`, `reshape`, and `transpose` to implement `ravel`, `swapaxes`, etc. And indeed use those mixins in `ndarray` itself.

For this also having a summary of base functions/methods would be very helpful.
-- Marten


I would definitely support writing more mixins and helper functions (either in NumPy, or externally) to make it easier to re-implement NumPy's public API. Certainly there is plenty of room to make it easier to leverage __array_ufunc__ and __array_function__.

For some recent examples of what these helpers functions could look like, see JAX's implementation of NumPy, which is written in terms of a much smaller array library called LAX:
https://github.com/google/jax/blob/9dfe27880517d5583048e7a3384b504681968fb4/jax/numpy/lax_numpy.py

Hypothetically, JAX could be written on top of a "restricted NumPy" instead, which in turn could have an implementation written in LAX. This would facilitate reusing JAX's higher level functions for automatic differentiation and vectorization on top of different array backends. 

I would also be happy to see guidance for NumPy API re-implementers, both for those scratching from scratch (e.g., in a new language) or who plan to copy NumPy's Python API (e.g., with __array_function__).

I would focus on:
1. Describing the tradeoffs of challenging design decisions that NumPy may have gotten wrong, e.g., scalars and indexing.
2. Describing common "gotchas" where it's easy to deviate from NumPy's semantics unintentionally, e.g., with scalar arithmetic dtypes or indexing edge cases.

I would *not* try to identify a "core" list of methods/functionality to implement. Everyone uses their own slice of NumPy's API, so the rational approach for anyone trying to reimplement exactly (i.e., with __array_function__) is to start with a minimal subset and add functionality on demand to meet user's needs. Also, many of the choices involved in making an array library don't really have objectively right or wrong answers, and authors are going to make intentional deviations from NumPy's semantics when it makes sense for them.

Cheers,
Stephan

 
 
_______________________________________________
NumPy-Discussion mailing list
NumPy-Discussion@python.org
https://mail.python.org/mailman/listinfo/numpy-discussion