advice for using datetime64 in C binding
Hello, I using the following code: if (PyArray_TYPE(arr1) == NPY_DATETIME) { // Ensure datetime64[ms] auto tmp = reinterpret_cast<PyArrayObject*>(PyObject_CallMethod(reinterpret_cast<PyObject*>(arr1), "astype", "(s)", "datetime64[ms]")); std::swap(arr1, tmp); Py_XDECREF(tmp); // Ensure integer tmp = reinterpret_cast<PyArrayObject*>(PyObject_CallMethod(reinterpret_cast<PyObject*>(arr1), "astype", "(s)", "i8")); std::swap(arr1, tmp); Py_XDECREF(tmp); tmp = reinterpret_cast<PyArrayObject*>(PyArray_FromArray(arr1, PyArray_DescrFromType(NPY_INT64), NPY_ARRAY_IN_ARRAY)); std::swap(arr1, tmp); Py_XDECREF(tmp); } First, if something is wrong with my code let me known. Then I wonder if I can have a safe shortcut to avoid converting datetime64 to i8. I guess the internal data of datetime64[ms] is i8 thus copying and casting the array may be avoided. Thanks by advance Best regards.
On Tue, 2022-01-18 at 14:56 +0100, Benoit Gschwind wrote:
Yes, you can assume datetime64 is stored as an i8 with the unit and possible byteswapping. Both of which, your initial cast will ensure. The internal data is i8, except for the special NaT value. Code-wise, you can avoid calling `astype`, but if you do (also in python), I suggest to pass `copy=False`, so that it does not copy if it clearly is not necessary (I am hoping to improve on the "clearly" here at some point). The last call again seems to be a no-op? Just the last call with the correct `datetime64[ms]` descriptor could be enough. Cheers, Sebastian
On Tue, 2022-01-18 at 18:29 +0100, Benoit Gschwind wrote:
Ah, yes, you are right, and it should indeed by very quick, anyway. My guess is, that you can do _only_ the last call with the appropriate `datetime64[ms]` descriptor passed in. (Since whatever C-code that follows is allowed to work with datetime64 as if it were int64.) Cheers, Sebastian
Hello, I found my ankser in own code ^^ static PyArray_Descr * create_datetime64_ms_dtype() { // Extrapolated from numpy sources PyArray_Descr * dtype = PyArray_DescrFromType(NPY_DATETIME); reinterpret_cast<PyArray_DatetimeDTypeMetaData *>(dtype->c_metadata)->meta.base = NPY_FR_ms; return dtype; } On Thu, 2022-01-20 at 09:20 +0100, Benoit Gschwind wrote:
On Fri, 2022-02-18 at 15:53 +0100, Benoit Gschwind wrote:
Oh sorry, yeah, I guess that works, I am not sure we have anything better in NumPy. The code itself is wrong though, you must make use of `PyArray_DescrNewFromType`: *never* mutate the result returned by `PyArray_DescrFromType`. After replacing, you should also check for `dtype == NULL` since errors could happen (not that it is relevant in practice, it only fails if Python fails to allocate that small descriptor object). Cheers, Sebastian
participants (2)
-
Benoit Gschwind -
Sebastian Berg