[Numpy-discussion] NEP: array API standard adoption (NEP 47)

Thu Mar 11 12:07:42 EST 2021

On Thu, 2021-03-11 at 12:37 +0100, Ralf Gommers wrote:
> On Wed, Mar 10, 2021 at 6:41 PM Sebastian Berg < 
> sebastian at sipsolutions.net>
> wrote:
> 
> > Top Posting, to discuss post specific questions about NEP 47 and
> > partially the start on implementing it in:
> > 
> >     https://github.com/numpy/numpy/pull/18585
> > 
> > There are probably many more that will crop up. But for me, each of
> > these is a pretty major difficulty without a clear answer as of
> > now.
> > 
> 
> All great questions, that Sebastian. Let me reply to the questions
> that
> Aaron didn't reply to inline below.
> 

To be clear, I do not expect complete answers to these questions right
now.  (Although being unsure about some of them does make me slightly
reluctant to merge the work-in-progress into NumPy proper as opposed to
a separate repo.)

Also, yes, most/all questions are hopefully are just trivialities to
check of (or no more than seeds for thought).  Or even just a starting
point for making NEP 47's "Usage and Impact" section more complete
including them as either "example usage patterns" or "limitations".

My second takeaway from the questions is that I have doubts the
"minimal" version will pan out, it feels like many of the questions
might disappear if you drop that part. So, from my current thinking,
the minimal implementation may not be a good "NEP 47" implementation.

That does _not_ mean that I think you should pause and reconsider or
even worry about pleasing me with good answers!  Just continue under
whatever assumption you prefer and if it turns out that "minimal" won't
work for NEP 47: no harm done!  We need a "minimal implementation" in
any case.

Cheers,

Sebastian

[1] If SciPy needs an additional NumPy code path to keep support
`object` arrays or other dtypes – right now even complex –, then the
reader needs to be aware of that to make a decision if NEP 47 will
actually help for their library.
Will AstroPy have to reimplement `astropy.units.Quantity` to be
"standard conform" (is that even possible!?) before it can easily adopt
it any of its API that currently works with `astropy.units.Quantity`?

> 
> > 1. I still need clarity how a library is supposed to use this
> > namespace
> > when the user passes in a NumPy array (mentioned before).  The user
> > must get back a NumPy array after all.  Maybe that is just a
> > decorator,
> > but it seems important.
> > 
> 
> I agree that it will be a common pattern that libraries will accept
> all
> standard-compliant array types plus numpy.ndarray. And the output
> array
> type should match the input type. In Aaron's implementation the new
> array
> object has a numpy.ndarray as private attribute, so that's the
> instance
> that should be returned. A decorator seems like a sensible way to
> handle
> that. Or a simple utility function, something like `return
> correct_arraytype(out)`.
> 
> Either way, that pattern should be added to NEP 47. I don't see a
> fundamental problem here, we just need to find the nicest UX for it.
> 
> 3. For all other function, the same problem applies. You don't
> actually
> > have anything to fix NumPy promotion rules.  You could bake your
> > own
> > cake here for numeric types, but I am not sure, you might also need
> > NEP
> > 43 in all its promotion power to pull it off.
> > 
> 
> This is probably the single most difficult question implementation-
> wise.
> Note that there are only numerical dtypes (plus boolean), so dealing
> with
> string, datetime, object or third-party dtypes is a non-issue.
> 
> 4. The PR makes no attempt at handling binary operators in any way
> > aside from greedily coercing the other operand.
> > 
> 
> Agreed. This is the same point as (3) I think - how to handle dtype
> promotion is the main open question.
>
> 
> > 5. What happens with a mix of array-likes or even array subclasses
> > like
> > `astropy.quantity`?
> > 
> 
> Array-likes (e.g. list) should raise an exception, the NEP clearly
> says "do
> not accept array_like dtypes". This is what every other array/tensor
> library already does.
> 
> Array subclasses should work as expected, assuming they're valid
> subclasses
> and not things like np.matrix. Using Mypy will help avoid writing
> more
> subclasses that break the Liskov substitution principle. More
> comments in
> https://numpy.org/neps/nep-0047-array-api-standard.html#the-asarray-asanyarray-pattern
> 
> Mixing two different types of arrays into a single function call
> should
> raise an exception. A design goal is: enable writing functions
> `somefunc(x1, x2)` that work for any type of array where `x1, x2`
> come from
> the same library = so they're either the same type, or two types for
> which
> the library itself knows how to mix them. If x1 and x2 are from
> different
> libraries, this will raise an exception.
> 
> To be clear, it is not intended that `np.array_api.somefunc(x_cupy)`
> works
> - this will raise an exception.
> 
> Cheers,
> Ralf
> 
> 
> 
> > 
> > I don't think we have to figure out everything up-front, but I do
> > think
> > there are a few very fundamental questions still open, at least for
> > me
> > personally.
> > 
> > Cheers,
> > 
> > Sebastian
> > 
> > 
> > 
> _______________________________________________
> NumPy-Discussion mailing list
> NumPy-Discussion at python.org
> https://mail.python.org/mailman/listinfo/numpy-discussion

-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 833 bytes
Desc: This is a digitally signed message part
URL: <https://mail.python.org/pipermail/numpy-discussion/attachments/20210311/62776a42/attachment.sig>