Hi, When registering a custom cast function from datetime64 to another dtype, how can I get the units? I am calling PyArray_RegisterCastFunc from NPY_DATETIME. Ideally, I'd like to register a separate cast function for each datetime64 units (or none at all... I don't want all units to be castable). Next best thing would be to obtain the units in the cast function and dispatch accordingly. Is this possible? I glanced through the code, and it looks like there's a lot of hard-coded logic around datetime64, but I didn't go through it carefully. Thought I'd ask before drilling further down. Thanks in advance, Alex
Hi Alex, On Mon, 2019-05-13 at 00:35 -0400, Alex Samuel wrote:
Hi,
When registering a custom cast function from datetime64 to another dtype, how can I get the units?
I am calling PyArray_RegisterCastFunc from NPY_DATETIME. Ideally, I'd like to register a separate cast function for each datetime64 units (or none at all... I don't want all units to be castable). Next best thing would be to obtain the units in the cast function and dispatch accordingly.
Is this possible? I glanced through the code, and it looks like there's a lot of hard-coded logic around datetime64, but I didn't go through it carefully. Thought I'd ask before drilling further down.
No, I do not think that is possible. But you do get the array pointers during the cast. I honestly would prefer not to promise that all of this will survive if we change this in numpy, since there are almost no users. But I think if we change it I could promise to help with cleaning it up in ora. I think this is public API (but I do not really like anyone using it ;)), so you can use: ``` /* * This function returns a pointer to the DateTimeMetaData * contained within the provided datetime dtype. */ static PyArray_DatetimeMetaData * get_datetime_metadata_from_dtype(PyArray_Descr *dtype) { /* original error check for DATETIME unnecessary for you */ return &(((PyArray_DatetimeDTypeMetaData *)dtype->c_metadata)->meta); } ``` And in the castfunc (fromarr is passed in as a void * as far as): ``` NPY_DATETIMEUNIT base = get_datetime_metadata_from_dtype( PyArray_DESCR(fromarr))->base; ``` Where NPY_DATETIMEUNIT is the enum defined in ndarraytypes.h. The logic will have to happen inside the cast func. I would hope there was a better way, but I cannot think of any, and I am scared that supporting this will add yet another ugly hack when we want to improve dtypes... Best, Sebastian
Thanks in advance, Alex
_______________________________________________ NumPy-Discussion mailing list NumPy-Discussion@python.org https://mail.python.org/mailman/listinfo/numpy-discussion
Thanks! I understand your hesitation; certainly if I am using undocumented APIs (or even if they are otherwise unused) I understand that it will be on my shoulders to preserve compatibility if you change things. But I would definitely like to get basic functionality working with current (and older, to be honest) NumPy versions. Ah, I missed previously that the array objects are passed as the void* args to the PyArray_VectorUnaryFunc. But it's documented here. https://docs.scipy.org/doc/numpy-1.13.0/reference/c-api.types-and-structures... (Why aren't these two args PyArrayObject* then?) As far as I can tell, get_datetime_metadata_from_dtype() isn't a public API? It certainly isn't spelled like one. It's declared in numpy/core/src/multiarray/_datetime.h, which doesn't appear to be #included in any other header. I suppose I could duplicate the struct layout in my own code to fish out the field I need. Obviously, compatibility is my own problem then. Regards, Alex On Mon, May 13, 2019, at 01:31, Sebastian Berg wrote:
Hi Alex,
On Mon, 2019-05-13 at 00:35 -0400, Alex Samuel wrote:
Hi,
When registering a custom cast function from datetime64 to another dtype, how can I get the units?
I am calling PyArray_RegisterCastFunc from NPY_DATETIME. Ideally, I'd like to register a separate cast function for each datetime64 units (or none at all... I don't want all units to be castable). Next best thing would be to obtain the units in the cast function and dispatch accordingly.
Is this possible? I glanced through the code, and it looks like there's a lot of hard-coded logic around datetime64, but I didn't go through it carefully. Thought I'd ask before drilling further down. get_datetime_metadata_from_dtype
No, I do not think that is possible. But you do get the array pointers during the cast. I honestly would prefer not to promise that all of this will survive if we change this in numpy, since there are almost no users. But I think if we change it I could promise to help with cleaning it up in ora.
I think this is public API (but I do not really like anyone using it ;)), so you can use:
``` /* * This function returns a pointer to the DateTimeMetaData * contained within the provided datetime dtype. */ static PyArray_DatetimeMetaData * get_datetime_metadata_from_dtype(PyArray_Descr *dtype) { /* original error check for DATETIME unnecessary for you */ return &(((PyArray_DatetimeDTypeMetaData *)dtype->c_metadata)->meta); } ```
And in the castfunc (fromarr is passed in as a void * as far as): ``` NPY_DATETIMEUNIT base = get_datetime_metadata_from_dtype( PyArray_DESCR(fromarr))->base; ```
Where NPY_DATETIMEUNIT is the enum defined in ndarraytypes.h. The logic will have to happen inside the cast func.
I would hope there was a better way, but I cannot think of any, and I am scared that supporting this will add yet another ugly hack when we want to improve dtypes...
Best,
Sebastian
Thanks in advance, Alex
_______________________________________________ NumPy-Discussion mailing list NumPy-Discussion@python.org https://mail.python.org/mailman/listinfo/numpy-discussion
_______________________________________________ NumPy-Discussion mailing list NumPy-Discussion@python.org https://mail.python.org/mailman/listinfo/numpy-discussion
*Attachments:* * signature.asc
participants (2)
-
Alex Samuel -
Sebastian Berg