[Numpy-discussion] defining a NumPy API standard?

[Stephan Hoyer]

On Sun, Jun 2, 2019 at 1:08 PM Marten van Kerkwijk <
m.h.vankerkwijk at gmail.com> wrote:

>
>
> On Sun, Jun 2, 2019 at 2:21 PM Eric Wieser <wieser.eric+numpy at gmail.com>
> wrote:
>
>> Some of your categories here sound like they might be suitable for ABCs
>> that provide mixin methods, which is something I think Hameer suggested in
>> the past. Perhaps it's worth re-exploring that avenue.
>>
>> Eric
>>
>>
> Indeed, and of course for __array_ufunc__ we moved there a bit already,
> with `NDArrayOperatorsMixin` [1].
> One could certainly similarly have NDShapingMixin that, e.g., relied on
> `shape`, `reshape`, and `transpose` to implement `ravel`, `swapaxes`, etc.
> And indeed use those mixins in `ndarray` itself.
>
> For this also having a summary of base functions/methods would be very
> helpful.
> -- Marten
>


I would definitely support writing more mixins and helper functions (either
in NumPy, or externally) to make it easier to re-implement NumPy's public
API. Certainly there is plenty of room to make it easier to leverage
__array_ufunc__ and __array_function__.

For some recent examples of what these helpers functions could look like,
see JAX's implementation of NumPy, which is written in terms of a much
smaller array library called LAX:
https://github.com/google/jax/blob/9dfe27880517d5583048e7a3384b504681968fb4/jax/numpy/lax_numpy.py

Hypothetically, JAX could be written on top of a "restricted NumPy"
instead, which in turn could have an implementation written in LAX. This
would facilitate reusing JAX's higher level functions for automatic
differentiation and vectorization on top of different array backends.

I would also be happy to see guidance for NumPy API re-implementers, both
for those scratching from scratch (e.g., in a new language) or who plan to
copy NumPy's Python API (e.g., with __array_function__).

I would focus on:
1. Describing the tradeoffs of challenging design decisions that NumPy may
have gotten wrong, e.g., scalars and indexing.
2. Describing common "gotchas" where it's easy to deviate from NumPy's
semantics unintentionally, e.g., with scalar arithmetic dtypes or indexing
edge cases.

I would *not* try to identify a "core" list of methods/functionality to
implement. Everyone uses their own slice of NumPy's API, so the rational
approach for anyone trying to reimplement exactly (i.e., with
__array_function__) is to start with a minimal subset and add functionality
on demand to meet user's needs. Also, many of the choices involved in
making an array library don't really have objectively right or wrong
answers, and authors are going to make intentional deviations from NumPy's
semantics when it makes sense for them.

Cheers,
Stephan



>
> _______________________________________________
> NumPy-Discussion mailing list
> NumPy-Discussion at python.org
> https://mail.python.org/mailman/listinfo/numpy-discussion
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/numpy-discussion/attachments/20190602/e2a0166f/attachment.html>