On Tue, 2021-03-16 at 13:17 -0500, Lee Johnston wrote:
> Is the work on NEP 42 custom DTypes far enough along to experiment
> with?
>

TL;DR:  Its not quite ready, but if we work together I think we could
experiment a fair bit.  Mainly ufuncs are still limited (though not
quite completely missing).  The main problem is that we need to find a
way to expose the currently private API.

I would be happy to discuss this also in a call.


** The long story: **

There is one more PR related to casting, for which merge should be
around the corner. And which would bring a lot bang to such an
experiment:

https://github.com/numpy/numpy/pull/18398


At that point, the new machinery supports (or is used for):

* Array-coercion: `np.array([your_scalar])` or
  `np.array([1], dtype=your_dtype)`.

* Casting (practically full support).

* UFuncs do not quite work. But short of writing `np.add(arr1, arr2)`
  with your DType involved, you can try a whole lot. (see below)

* Promotion `np.result_type` should work very soon, but probably isn't
  is not very relevant anyway until ufuncs are fully implemented.

That should allow you to do a lot of good experimentation, but due to
the ufunc limitation, maybe not well on "existing" python code.


The long story about limitations is:

We are missing exposure of the new public API.  I think I should be
able to provide a solution for this pretty quickly, but it might
require working of a NumPy branch.  (I will write another email about
it, hopefully we can find a better solution.)


Limitations for UFuncs:  UFuncs are the next big project, so to try it
fully you will need some patience, unfortunately.

But, there is some good news!  You can write most of the "ufunc"
already, you just can't "register" it.
So what I can already offer you is a "DType-specific UFunc", e.g.:

   unit_dtype_multiply(np.array([1.], dtype=Float64UnitDType("m")),
                       np.array([2.], dtype=Float64UnitDtype("s")))

And get out `np.array([2.], dtype=Float64UnitDtype("m s"))`.

But you can't write `np.multiple(arr1, arr2)` or `arr1 * arr2` yet.
Both registration and "promotion" logic are missing.

I admit promotion may be one of the trickiest things, but trying this a
bit might help with getting a clearer picture for promotion as well.


The main last limitation is that I did not replace or create "fallback"
solutions and/or replacement for the legacy `dtype->f-><slots>` yet.
This is not a serious limitation for experimentation, though.  It might
even make sense to keep some of them around and replace them slowly.


And of course, all the small issues/limitations that are not fixed
because nobody tried yet...



I hope this doesn't scare you away, or at least not for long :/.  It
could be very useful to start experimentation soon to push things
forward a bit quicker.  And I really want to have at least an
experimental version in NumPy 1.21.

Cheers,

Sebastian


> Lee
> _______________________________________________
> NumPy-Discussion mailing list
> NumPy-Discussion@python.org
> https://mail.python.org/mailman/listinfo/numpy-discussion

_______________________________________________
NumPy-Discussion mailing list
NumPy-Discussion@python.org
https://mail.python.org/mailman/listinfo/numpy-discussion