advice for using datetime64 in C binding
Hello, I using the following code: if (PyArray_TYPE(arr1) == NPY_DATETIME) { // Ensure datetime64[ms] auto tmp = reinterpret_cast<PyArrayObject*>(PyObject_CallMethod(reinterpret_cast<PyObject*>(arr1), "astype", "(s)", "datetime64[ms]")); std::swap(arr1, tmp); Py_XDECREF(tmp); // Ensure integer tmp = reinterpret_cast<PyArrayObject*>(PyObject_CallMethod(reinterpret_cast<PyObject*>(arr1), "astype", "(s)", "i8")); std::swap(arr1, tmp); Py_XDECREF(tmp); tmp = reinterpret_cast<PyArrayObject*>(PyArray_FromArray(arr1, PyArray_DescrFromType(NPY_INT64), NPY_ARRAY_IN_ARRAY)); std::swap(arr1, tmp); Py_XDECREF(tmp); } First, if something is wrong with my code let me known. Then I wonder if I can have a safe shortcut to avoid converting datetime64 to i8. I guess the internal data of datetime64[ms] is i8 thus copying and casting the array may be avoided. Thanks by advance Best regards.
On Tue, 2022-01-18 at 14:56 +0100, Benoit Gschwind wrote:
Hello,
I using the following code:
if (PyArray_TYPE(arr1) == NPY_DATETIME) { // Ensure datetime64[ms] auto tmp = reinterpret_cast<PyArrayObject*>(PyObject_CallMethod(reinterpret_cast <PyObject*>(arr1), "astype", "(s)", "datetime64[ms]")); std::swap(arr1, tmp); Py_XDECREF(tmp); // Ensure integer tmp = reinterpret_cast<PyArrayObject*>(PyObject_CallMethod(reinterpret_cast <PyObject*>(arr1), "astype", "(s)", "i8")); std::swap(arr1, tmp); Py_XDECREF(tmp); tmp = reinterpret_cast<PyArrayObject*>(PyArray_FromArray(arr1, PyArray_DescrFromType(NPY_INT64), NPY_ARRAY_IN_ARRAY)); std::swap(arr1, tmp); Py_XDECREF(tmp); }
First, if something is wrong with my code let me known. Then I wonder if I can have a safe shortcut to avoid converting datetime64 to i8. I guess the internal data of datetime64[ms] is i8 thus copying and casting the array may be avoided.
Yes, you can assume datetime64 is stored as an i8 with the unit and possible byteswapping. Both of which, your initial cast will ensure. The internal data is i8, except for the special NaT value. Code-wise, you can avoid calling `astype`, but if you do (also in python), I suggest to pass `copy=False`, so that it does not copy if it clearly is not necessary (I am hoping to improve on the "clearly" here at some point). The last call again seems to be a no-op? Just the last call with the correct `datetime64[ms]` descriptor could be enough. Cheers, Sebastian
Thanks by advance Best regards.
_______________________________________________ NumPy-Discussion mailing list -- numpy-discussion@python.org To unsubscribe send an email to numpy-discussion-leave@python.org https://mail.python.org/mailman3/lists/numpy-discussion.python.org/ Member address: sebastian@sipsolutions.net
Hello Sebastian, Thanks for detail. The last call has the NPY_ARRAY_C_CONTIGUOUS and NPY_ARRAY_ALIGNED, thus I guess it can be no-op most of the time but for some unknown case it's may be safer ? Best regards On Tue, 2022-01-18 at 09:22 -0600, Sebastian Berg wrote:
On Tue, 2022-01-18 at 14:56 +0100, Benoit Gschwind wrote:
Hello,
I using the following code:
if (PyArray_TYPE(arr1) == NPY_DATETIME) { // Ensure datetime64[ms] auto tmp = reinterpret_cast<PyArrayObject*>(PyObject_CallMethod(reinterpret_ca st <PyObject*>(arr1), "astype", "(s)", "datetime64[ms]")); std::swap(arr1, tmp); Py_XDECREF(tmp); // Ensure integer tmp = reinterpret_cast<PyArrayObject*>(PyObject_CallMethod(reinterpret_ca st <PyObject*>(arr1), "astype", "(s)", "i8")); std::swap(arr1, tmp); Py_XDECREF(tmp); tmp = reinterpret_cast<PyArrayObject*>(PyArray_FromArray(arr1, PyArray_DescrFromType(NPY_INT64), NPY_ARRAY_IN_ARRAY)); std::swap(arr1, tmp); Py_XDECREF(tmp); }
First, if something is wrong with my code let me known. Then I wonder if I can have a safe shortcut to avoid converting datetime64 to i8. I guess the internal data of datetime64[ms] is i8 thus copying and casting the array may be avoided.
Yes, you can assume datetime64 is stored as an i8 with the unit and possible byteswapping. Both of which, your initial cast will ensure.
The internal data is i8, except for the special NaT value.
Code-wise, you can avoid calling `astype`, but if you do (also in python), I suggest to pass `copy=False`, so that it does not copy if it clearly is not necessary (I am hoping to improve on the "clearly" here at some point).
The last call again seems to be a no-op? Just the last call with the correct `datetime64[ms]` descriptor could be enough.
Cheers,
Sebastian
Thanks by advance Best regards.
_______________________________________________ NumPy-Discussion mailing list -- numpy-discussion@python.org To unsubscribe send an email to numpy-discussion-leave@python.org https://mail.python.org/mailman3/lists/numpy-discussion.python.org/ Member address: sebastian@sipsolutions.net
_______________________________________________ NumPy-Discussion mailing list -- numpy-discussion@python.org To unsubscribe send an email to numpy-discussion-leave@python.org https://mail.python.org/mailman3/lists/numpy-discussion.python.org/ Member address: gschwind@gnu-log.net
On Tue, 2022-01-18 at 18:29 +0100, Benoit Gschwind wrote:
Hello Sebastian,
Thanks for detail.
The last call has the NPY_ARRAY_C_CONTIGUOUS and NPY_ARRAY_ALIGNED, thus I guess it can be no-op most of the time but for some unknown case it's may be safer ?
Ah, yes, you are right, and it should indeed by very quick, anyway. My guess is, that you can do _only_ the last call with the appropriate `datetime64[ms]` descriptor passed in. (Since whatever C-code that follows is allowed to work with datetime64 as if it were int64.) Cheers, Sebastian
Best regards
On Tue, 2022-01-18 at 09:22 -0600, Sebastian Berg wrote:
On Tue, 2022-01-18 at 14:56 +0100, Benoit Gschwind wrote:
Hello,
I using the following code:
if (PyArray_TYPE(arr1) == NPY_DATETIME) { // Ensure datetime64[ms] auto tmp = reinterpret_cast<PyArrayObject*>(PyObject_CallMethod(reinterpret_ ca st <PyObject*>(arr1), "astype", "(s)", "datetime64[ms]")); std::swap(arr1, tmp); Py_XDECREF(tmp); // Ensure integer tmp = reinterpret_cast<PyArrayObject*>(PyObject_CallMethod(reinterpret_ ca st <PyObject*>(arr1), "astype", "(s)", "i8")); std::swap(arr1, tmp); Py_XDECREF(tmp); tmp = reinterpret_cast<PyArrayObject*>(PyArray_FromArray(arr1, PyArray_DescrFromType(NPY_INT64), NPY_ARRAY_IN_ARRAY)); std::swap(arr1, tmp); Py_XDECREF(tmp); }
First, if something is wrong with my code let me known. Then I wonder if I can have a safe shortcut to avoid converting datetime64 to i8. I guess the internal data of datetime64[ms] is i8 thus copying and casting the array may be avoided.
Yes, you can assume datetime64 is stored as an i8 with the unit and possible byteswapping. Both of which, your initial cast will ensure.
The internal data is i8, except for the special NaT value.
Code-wise, you can avoid calling `astype`, but if you do (also in python), I suggest to pass `copy=False`, so that it does not copy if it clearly is not necessary (I am hoping to improve on the "clearly" here at some point).
The last call again seems to be a no-op? Just the last call with the correct `datetime64[ms]` descriptor could be enough.
Cheers,
Sebastian
Thanks by advance Best regards.
_______________________________________________ NumPy-Discussion mailing list -- numpy-discussion@python.org To unsubscribe send an email to numpy-discussion-leave@python.org https://mail.python.org/mailman3/lists/numpy-discussion.python.org/ Member address: sebastian@sipsolutions.net
_______________________________________________ NumPy-Discussion mailing list -- numpy-discussion@python.org To unsubscribe send an email to numpy-discussion-leave@python.org https://mail.python.org/mailman3/lists/numpy-discussion.python.org/ Member address: gschwind@gnu-log.net
_______________________________________________ NumPy-Discussion mailing list -- numpy-discussion@python.org To unsubscribe send an email to numpy-discussion-leave@python.org https://mail.python.org/mailman3/lists/numpy-discussion.python.org/ Member address: sebastian@sipsolutions.net
Hello Sebastian, Thanks for the precision Best regards On Tue, 2022-01-18 at 11:52 -0600, Sebastian Berg wrote:
On Tue, 2022-01-18 at 18:29 +0100, Benoit Gschwind wrote:
Hello Sebastian,
Thanks for detail.
The last call has the NPY_ARRAY_C_CONTIGUOUS and NPY_ARRAY_ALIGNED, thus I guess it can be no-op most of the time but for some unknown case it's may be safer ?
Ah, yes, you are right, and it should indeed by very quick, anyway. My guess is, that you can do _only_ the last call with the appropriate `datetime64[ms]` descriptor passed in. (Since whatever C-code that follows is allowed to work with datetime64 as if it were int64.)
Cheers,
Sebastian
Best regards
On Tue, 2022-01-18 at 09:22 -0600, Sebastian Berg wrote:
On Tue, 2022-01-18 at 14:56 +0100, Benoit Gschwind wrote:
Hello,
I using the following code:
if (PyArray_TYPE(arr1) == NPY_DATETIME) { // Ensure datetime64[ms] auto tmp = reinterpret_cast<PyArrayObject*>(PyObject_CallMethod(reinterpre t_ ca st <PyObject*>(arr1), "astype", "(s)", "datetime64[ms]")); std::swap(arr1, tmp); Py_XDECREF(tmp); // Ensure integer tmp = reinterpret_cast<PyArrayObject*>(PyObject_CallMethod(reinterpre t_ ca st <PyObject*>(arr1), "astype", "(s)", "i8")); std::swap(arr1, tmp); Py_XDECREF(tmp); tmp = reinterpret_cast<PyArrayObject*>(PyArray_FromArray(arr1, PyArray_DescrFromType(NPY_INT64), NPY_ARRAY_IN_ARRAY)); std::swap(arr1, tmp); Py_XDECREF(tmp); }
First, if something is wrong with my code let me known. Then I wonder if I can have a safe shortcut to avoid converting datetime64 to i8. I guess the internal data of datetime64[ms] is i8 thus copying and casting the array may be avoided.
Yes, you can assume datetime64 is stored as an i8 with the unit and possible byteswapping. Both of which, your initial cast will ensure.
The internal data is i8, except for the special NaT value.
Code-wise, you can avoid calling `astype`, but if you do (also in python), I suggest to pass `copy=False`, so that it does not copy if it clearly is not necessary (I am hoping to improve on the "clearly" here at some point).
The last call again seems to be a no-op? Just the last call with the correct `datetime64[ms]` descriptor could be enough.
Cheers,
Sebastian
Thanks by advance Best regards.
_______________________________________________ NumPy-Discussion mailing list -- numpy-discussion@python.org To unsubscribe send an email to numpy-discussion-leave@python.org https://mail.python.org/mailman3/lists/numpy-discussion.python.org/ Member address: sebastian@sipsolutions.net
_______________________________________________ NumPy-Discussion mailing list -- numpy-discussion@python.org To unsubscribe send an email to numpy-discussion-leave@python.org https://mail.python.org/mailman3/lists/numpy-discussion.python.org/ Member address: gschwind@gnu-log.net
_______________________________________________ NumPy-Discussion mailing list -- numpy-discussion@python.org To unsubscribe send an email to numpy-discussion-leave@python.org https://mail.python.org/mailman3/lists/numpy-discussion.python.org/ Member address: sebastian@sipsolutions.net
_______________________________________________ NumPy-Discussion mailing list -- numpy-discussion@python.org To unsubscribe send an email to numpy-discussion-leave@python.org https://mail.python.org/mailman3/lists/numpy-discussion.python.org/ Member address: gschwind@gnu-log.net
Hello, How can I create np.dtype("datetime64[ms]") in C to get the coresponding PyArray_Descr ? Thank you by advance On Wed, 2022-01-19 at 10:08 +0100, Benoit Gschwind wrote:
Hello Sebastian,
Thanks for the precision
Best regards
On Tue, 2022-01-18 at 11:52 -0600, Sebastian Berg wrote:
On Tue, 2022-01-18 at 18:29 +0100, Benoit Gschwind wrote:
Hello Sebastian,
Thanks for detail.
The last call has the NPY_ARRAY_C_CONTIGUOUS and NPY_ARRAY_ALIGNED, thus I guess it can be no-op most of the time but for some unknown case it's may be safer ?
Ah, yes, you are right, and it should indeed by very quick, anyway. My guess is, that you can do _only_ the last call with the appropriate `datetime64[ms]` descriptor passed in. (Since whatever C-code that follows is allowed to work with datetime64 as if it were int64.)
Cheers,
Sebastian
Best regards
On Tue, 2022-01-18 at 09:22 -0600, Sebastian Berg wrote:
On Tue, 2022-01-18 at 14:56 +0100, Benoit Gschwind wrote:
Hello,
I using the following code:
if (PyArray_TYPE(arr1) == NPY_DATETIME) { // Ensure datetime64[ms] auto tmp = reinterpret_cast<PyArrayObject*>(PyObject_CallMethod(reinterp re t_ ca st <PyObject*>(arr1), "astype", "(s)", "datetime64[ms]")); std::swap(arr1, tmp); Py_XDECREF(tmp); // Ensure integer tmp = reinterpret_cast<PyArrayObject*>(PyObject_CallMethod(reinterp re t_ ca st <PyObject*>(arr1), "astype", "(s)", "i8")); std::swap(arr1, tmp); Py_XDECREF(tmp); tmp = reinterpret_cast<PyArrayObject*>(PyArray_FromArray(arr1, PyArray_DescrFromType(NPY_INT64), NPY_ARRAY_IN_ARRAY)); std::swap(arr1, tmp); Py_XDECREF(tmp); }
First, if something is wrong with my code let me known. Then I wonder if I can have a safe shortcut to avoid converting datetime64 to i8. I guess the internal data of datetime64[ms] is i8 thus copying and casting the array may be avoided.
Yes, you can assume datetime64 is stored as an i8 with the unit and possible byteswapping. Both of which, your initial cast will ensure.
The internal data is i8, except for the special NaT value.
Code-wise, you can avoid calling `astype`, but if you do (also in python), I suggest to pass `copy=False`, so that it does not copy if it clearly is not necessary (I am hoping to improve on the "clearly" here at some point).
The last call again seems to be a no-op? Just the last call with the correct `datetime64[ms]` descriptor could be enough.
Cheers,
Sebastian
Thanks by advance Best regards.
_______________________________________________ NumPy-Discussion mailing list -- numpy-discussion@python.org To unsubscribe send an email to numpy-discussion-leave@python.org https://mail.python.org/mailman3/lists/numpy-discussion.python.org/ Member address: sebastian@sipsolutions.net
_______________________________________________ NumPy-Discussion mailing list -- numpy-discussion@python.org To unsubscribe send an email to numpy-discussion-leave@python.org https://mail.python.org/mailman3/lists/numpy-discussion.python.org/ Member address: gschwind@gnu-log.net
_______________________________________________ NumPy-Discussion mailing list -- numpy-discussion@python.org To unsubscribe send an email to numpy-discussion-leave@python.org https://mail.python.org/mailman3/lists/numpy-discussion.python.org/ Member address: sebastian@sipsolutions.net
_______________________________________________ NumPy-Discussion mailing list -- numpy-discussion@python.org To unsubscribe send an email to numpy-discussion-leave@python.org https://mail.python.org/mailman3/lists/numpy-discussion.python.org/ Member address: gschwind@gnu-log.net
_______________________________________________ NumPy-Discussion mailing list -- numpy-discussion@python.org To unsubscribe send an email to numpy-discussion-leave@python.org https://mail.python.org/mailman3/lists/numpy-discussion.python.org/ Member address: gschwind@gnu-log.net
Hello, I found my ankser in own code ^^ static PyArray_Descr * create_datetime64_ms_dtype() { // Extrapolated from numpy sources PyArray_Descr * dtype = PyArray_DescrFromType(NPY_DATETIME); reinterpret_cast<PyArray_DatetimeDTypeMetaData *>(dtype->c_metadata)->meta.base = NPY_FR_ms; return dtype; } On Thu, 2022-01-20 at 09:20 +0100, Benoit Gschwind wrote:
Hello,
How can I create np.dtype("datetime64[ms]") in C to get the coresponding PyArray_Descr ?
Thank you by advance
On Wed, 2022-01-19 at 10:08 +0100, Benoit Gschwind wrote:
Hello Sebastian,
Thanks for the precision
Best regards
On Tue, 2022-01-18 at 11:52 -0600, Sebastian Berg wrote:
On Tue, 2022-01-18 at 18:29 +0100, Benoit Gschwind wrote:
Hello Sebastian,
Thanks for detail.
The last call has the NPY_ARRAY_C_CONTIGUOUS and NPY_ARRAY_ALIGNED, thus I guess it can be no-op most of the time but for some unknown case it's may be safer ?
Ah, yes, you are right, and it should indeed by very quick, anyway. My guess is, that you can do _only_ the last call with the appropriate `datetime64[ms]` descriptor passed in. (Since whatever C-code that follows is allowed to work with datetime64 as if it were int64.)
Cheers,
Sebastian
Best regards
On Tue, 2022-01-18 at 09:22 -0600, Sebastian Berg wrote:
On Tue, 2022-01-18 at 14:56 +0100, Benoit Gschwind wrote:
Hello,
I using the following code:
if (PyArray_TYPE(arr1) == NPY_DATETIME) { // Ensure datetime64[ms] auto tmp = reinterpret_cast<PyArrayObject*>(PyObject_CallMethod(reinterp re t_ ca st <PyObject*>(arr1), "astype", "(s)", "datetime64[ms]")); std::swap(arr1, tmp); Py_XDECREF(tmp); // Ensure integer tmp = reinterpret_cast<PyArrayObject*>(PyObject_CallMethod(reinterp re t_ ca st <PyObject*>(arr1), "astype", "(s)", "i8")); std::swap(arr1, tmp); Py_XDECREF(tmp); tmp = reinterpret_cast<PyArrayObject*>(PyArray_FromArray(arr1, PyArray_DescrFromType(NPY_INT64), NPY_ARRAY_IN_ARRAY)); std::swap(arr1, tmp); Py_XDECREF(tmp); }
First, if something is wrong with my code let me known. Then I wonder if I can have a safe shortcut to avoid converting datetime64 to i8. I guess the internal data of datetime64[ms] is i8 thus copying and casting the array may be avoided.
Yes, you can assume datetime64 is stored as an i8 with the unit and possible byteswapping. Both of which, your initial cast will ensure.
The internal data is i8, except for the special NaT value.
Code-wise, you can avoid calling `astype`, but if you do (also in python), I suggest to pass `copy=False`, so that it does not copy if it clearly is not necessary (I am hoping to improve on the "clearly" here at some point).
The last call again seems to be a no-op? Just the last call with the correct `datetime64[ms]` descriptor could be enough.
Cheers,
Sebastian
Thanks by advance Best regards.
_______________________________________________ NumPy-Discussion mailing list -- numpy-discussion@python.org To unsubscribe send an email to numpy-discussion-leave@python.org https://mail.python.org/mailman3/lists/numpy-discussion.python.org/ Member address: sebastian@sipsolutions.net
_______________________________________________ NumPy-Discussion mailing list -- numpy-discussion@python.org To unsubscribe send an email to numpy-discussion-leave@python.org https://mail.python.org/mailman3/lists/numpy-discussion.python.org/ Member address: gschwind@gnu-log.net
_______________________________________________ NumPy-Discussion mailing list -- numpy-discussion@python.org To unsubscribe send an email to numpy-discussion-leave@python.org https://mail.python.org/mailman3/lists/numpy-discussion.python.org/ Member address: sebastian@sipsolutions.net
_______________________________________________ NumPy-Discussion mailing list -- numpy-discussion@python.org To unsubscribe send an email to numpy-discussion-leave@python.org https://mail.python.org/mailman3/lists/numpy-discussion.python.org/ Member address: gschwind@gnu-log.net
_______________________________________________ NumPy-Discussion mailing list -- numpy-discussion@python.org To unsubscribe send an email to numpy-discussion-leave@python.org https://mail.python.org/mailman3/lists/numpy-discussion.python.org/ Member address: gschwind@gnu-log.net
_______________________________________________ NumPy-Discussion mailing list -- numpy-discussion@python.org To unsubscribe send an email to numpy-discussion-leave@python.org https://mail.python.org/mailman3/lists/numpy-discussion.python.org/ Member address: gschwind@gnu-log.net
On Fri, 2022-02-18 at 15:53 +0100, Benoit Gschwind wrote:
Hello,
I found my ankser in own code ^^
static PyArray_Descr * create_datetime64_ms_dtype() { // Extrapolated from numpy sources PyArray_Descr * dtype = PyArray_DescrFromType(NPY_DATETIME); reinterpret_cast<PyArray_DatetimeDTypeMetaData *>(dtype-
c_metadata)->meta.base = NPY_FR_ms; return dtype; }
Oh sorry, yeah, I guess that works, I am not sure we have anything better in NumPy. The code itself is wrong though, you must make use of `PyArray_DescrNewFromType`: *never* mutate the result returned by `PyArray_DescrFromType`. After replacing, you should also check for `dtype == NULL` since errors could happen (not that it is relevant in practice, it only fails if Python fails to allocate that small descriptor object). Cheers, Sebastian
On Thu, 2022-01-20 at 09:20 +0100, Benoit Gschwind wrote:
Hello,
How can I create np.dtype("datetime64[ms]") in C to get the coresponding PyArray_Descr ?
Thank you by advance
On Wed, 2022-01-19 at 10:08 +0100, Benoit Gschwind wrote:
Hello Sebastian,
Thanks for the precision
Best regards
On Tue, 2022-01-18 at 11:52 -0600, Sebastian Berg wrote:
On Tue, 2022-01-18 at 18:29 +0100, Benoit Gschwind wrote:
Hello Sebastian,
Thanks for detail.
The last call has the NPY_ARRAY_C_CONTIGUOUS and NPY_ARRAY_ALIGNED, thus I guess it can be no-op most of the time but for some unknown case it's may be safer ?
Ah, yes, you are right, and it should indeed by very quick, anyway. My guess is, that you can do _only_ the last call with the appropriate `datetime64[ms]` descriptor passed in. (Since whatever C-code that follows is allowed to work with datetime64 as if it were int64.)
Cheers,
Sebastian
Best regards
On Tue, 2022-01-18 at 09:22 -0600, Sebastian Berg wrote:
On Tue, 2022-01-18 at 14:56 +0100, Benoit Gschwind wrote: > Hello, > > I using the following code: > > if (PyArray_TYPE(arr1) == NPY_DATETIME) { > // Ensure datetime64[ms] > auto tmp = > reinterpret_cast<PyArrayObject*>(PyObject_CallMethod(rein > terp > re > t_ > ca > st > <PyObject*>(arr1), "astype", "(s)", "datetime64[ms]")); > std::swap(arr1, tmp); > Py_XDECREF(tmp); > // Ensure integer > tmp = > reinterpret_cast<PyArrayObject*>(PyObject_CallMethod(rein > terp > re > t_ > ca > st > <PyObject*>(arr1), "astype", "(s)", "i8")); > std::swap(arr1, tmp); > Py_XDECREF(tmp); > tmp = > reinterpret_cast<PyArrayObject*>(PyArray_FromArray(arr1, > PyArray_DescrFromType(NPY_INT64), NPY_ARRAY_IN_ARRAY)); > std::swap(arr1, tmp); > Py_XDECREF(tmp); > } > > First, if something is wrong with my code let me known. > Then > I > wonder > if I can have a safe shortcut to avoid converting > datetime64 > to > i8. > I > guess the internal data of datetime64[ms] is i8 thus > copying > and > casting the array may be avoided.
Yes, you can assume datetime64 is stored as an i8 with the unit and possible byteswapping. Both of which, your initial cast will ensure.
The internal data is i8, except for the special NaT value.
Code-wise, you can avoid calling `astype`, but if you do (also in python), I suggest to pass `copy=False`, so that it does not copy if it clearly is not necessary (I am hoping to improve on the "clearly" here at some point).
The last call again seems to be a no-op? Just the last call with the correct `datetime64[ms]` descriptor could be enough.
Cheers,
Sebastian
> > Thanks by advance > Best regards. > > > _______________________________________________ > NumPy-Discussion mailing list -- > numpy-discussion@python.org > To unsubscribe send an email to > numpy-discussion-leave@python.org > https://mail.python.org/mailman3/lists/numpy-discussion.python.org/ > Member address: sebastian@sipsolutions.net >
_______________________________________________ NumPy-Discussion mailing list -- numpy-discussion@python.org To unsubscribe send an email to numpy-discussion-leave@python.org https://mail.python.org/mailman3/lists/numpy-discussion.python.org/ Member address: gschwind@gnu-log.net
_______________________________________________ NumPy-Discussion mailing list -- numpy-discussion@python.org To unsubscribe send an email to numpy-discussion-leave@python.org https://mail.python.org/mailman3/lists/numpy-discussion.python.org/ Member address: sebastian@sipsolutions.net
_______________________________________________ NumPy-Discussion mailing list -- numpy-discussion@python.org To unsubscribe send an email to numpy-discussion-leave@python.org https://mail.python.org/mailman3/lists/numpy-discussion.python.org/ Member address: gschwind@gnu-log.net
_______________________________________________ NumPy-Discussion mailing list -- numpy-discussion@python.org To unsubscribe send an email to numpy-discussion-leave@python.org https://mail.python.org/mailman3/lists/numpy-discussion.python.org/ Member address: gschwind@gnu-log.net
_______________________________________________ NumPy-Discussion mailing list -- numpy-discussion@python.org To unsubscribe send an email to numpy-discussion-leave@python.org https://mail.python.org/mailman3/lists/numpy-discussion.python.org/ Member address: gschwind@gnu-log.net
_______________________________________________ NumPy-Discussion mailing list -- numpy-discussion@python.org To unsubscribe send an email to numpy-discussion-leave@python.org https://mail.python.org/mailman3/lists/numpy-discussion.python.org/ Member address: sebastian@sipsolutions.net
Thank you for fixing my code ! :) On Fri, 2022-02-18 at 09:00 -0600, Sebastian Berg wrote:
On Fri, 2022-02-18 at 15:53 +0100, Benoit Gschwind wrote:
Hello,
I found my ankser in own code ^^
static PyArray_Descr * create_datetime64_ms_dtype() { // Extrapolated from numpy sources PyArray_Descr * dtype = PyArray_DescrFromType(NPY_DATETIME); reinterpret_cast<PyArray_DatetimeDTypeMetaData *>(dtype-
c_metadata)->meta.base = NPY_FR_ms; return dtype; }
Oh sorry, yeah, I guess that works, I am not sure we have anything better in NumPy.
The code itself is wrong though, you must make use of `PyArray_DescrNewFromType`: *never* mutate the result returned by `PyArray_DescrFromType`.
After replacing, you should also check for `dtype == NULL` since errors could happen (not that it is relevant in practice, it only fails if Python fails to allocate that small descriptor object).
Cheers,
Sebastian
On Thu, 2022-01-20 at 09:20 +0100, Benoit Gschwind wrote:
Hello,
How can I create np.dtype("datetime64[ms]") in C to get the coresponding PyArray_Descr ?
Thank you by advance
On Wed, 2022-01-19 at 10:08 +0100, Benoit Gschwind wrote:
Hello Sebastian,
Thanks for the precision
Best regards
On Tue, 2022-01-18 at 11:52 -0600, Sebastian Berg wrote:
On Tue, 2022-01-18 at 18:29 +0100, Benoit Gschwind wrote:
Hello Sebastian,
Thanks for detail.
The last call has the NPY_ARRAY_C_CONTIGUOUS and NPY_ARRAY_ALIGNED, thus I guess it can be no-op most of the time but for some unknown case it's may be safer ?
Ah, yes, you are right, and it should indeed by very quick, anyway. My guess is, that you can do _only_ the last call with the appropriate `datetime64[ms]` descriptor passed in. (Since whatever C-code that follows is allowed to work with datetime64 as if it were int64.)
Cheers,
Sebastian
Best regards
On Tue, 2022-01-18 at 09:22 -0600, Sebastian Berg wrote: > On Tue, 2022-01-18 at 14:56 +0100, Benoit Gschwind wrote: > > Hello, > > > > I using the following code: > > > > if (PyArray_TYPE(arr1) == NPY_DATETIME) { > > // Ensure datetime64[ms] > > auto tmp = > > reinterpret_cast<PyArrayObject*>(PyObject_CallMethod(re > > in > > terp > > re > > t_ > > ca > > st > > <PyObject*>(arr1), "astype", "(s)", "datetime64[ms]")); > > std::swap(arr1, tmp); > > Py_XDECREF(tmp); > > // Ensure integer > > tmp = > > reinterpret_cast<PyArrayObject*>(PyObject_CallMethod(re > > in > > terp > > re > > t_ > > ca > > st > > <PyObject*>(arr1), "astype", "(s)", "i8")); > > std::swap(arr1, tmp); > > Py_XDECREF(tmp); > > tmp = > > reinterpret_cast<PyArrayObject*>(PyArray_FromArray(arr1 > > , > > PyArray_DescrFromType(NPY_INT64), NPY_ARRAY_IN_ARRAY)); > > std::swap(arr1, tmp); > > Py_XDECREF(tmp); > > } > > > > First, if something is wrong with my code let me known. > > Then > > I > > wonder > > if I can have a safe shortcut to avoid converting > > datetime64 > > to > > i8. > > I > > guess the internal data of datetime64[ms] is i8 thus > > copying > > and > > casting the array may be avoided. > > Yes, you can assume datetime64 is stored as an i8 with > the > unit > and > possible byteswapping. Both of which, your initial cast > will > ensure. > > The internal data is i8, except for the special NaT > value. > > Code-wise, you can avoid calling `astype`, but if you do > (also > in > python), I suggest to pass `copy=False`, so that it does > not > copy > if > it > clearly is not necessary (I am hoping to improve on the > "clearly" > here > at some point). > > The last call again seems to be a no-op? Just the last > call > with > the > correct `datetime64[ms]` descriptor could be enough. > > Cheers, > > Sebastian > > > > > > Thanks by advance > > Best regards. > > > > > > _______________________________________________ > > NumPy-Discussion mailing list -- > > numpy-discussion@python.org > > To unsubscribe send an email to > > numpy-discussion-leave@python.org > > https://mail.python.org/mailman3/lists/numpy-discussion.python.org/ > > Member address: sebastian@sipsolutions.net > > > > _______________________________________________ > NumPy-Discussion mailing list -- > numpy-discussion@python.org > To unsubscribe send an email to > numpy-discussion-leave@python.org > https://mail.python.org/mailman3/lists/numpy-discussion.python.org/ > Member address: gschwind@gnu-log.net
_______________________________________________ NumPy-Discussion mailing list -- numpy-discussion@python.org To unsubscribe send an email to numpy-discussion-leave@python.org https://mail.python.org/mailman3/lists/numpy-discussion.python.org/ Member address: sebastian@sipsolutions.net
_______________________________________________ NumPy-Discussion mailing list -- numpy-discussion@python.org To unsubscribe send an email to numpy-discussion-leave@python.org https://mail.python.org/mailman3/lists/numpy-discussion.python.org/ Member address: gschwind@gnu-log.net
_______________________________________________ NumPy-Discussion mailing list -- numpy-discussion@python.org To unsubscribe send an email to numpy-discussion-leave@python.org https://mail.python.org/mailman3/lists/numpy-discussion.python.org/ Member address: gschwind@gnu-log.net
_______________________________________________ NumPy-Discussion mailing list -- numpy-discussion@python.org To unsubscribe send an email to numpy-discussion-leave@python.org https://mail.python.org/mailman3/lists/numpy-discussion.python.org/ Member address: gschwind@gnu-log.net
_______________________________________________ NumPy-Discussion mailing list -- numpy-discussion@python.org To unsubscribe send an email to numpy-discussion-leave@python.org https://mail.python.org/mailman3/lists/numpy-discussion.python.org/ Member address: sebastian@sipsolutions.net
_______________________________________________ NumPy-Discussion mailing list -- numpy-discussion@python.org To unsubscribe send an email to numpy-discussion-leave@python.org https://mail.python.org/mailman3/lists/numpy-discussion.python.org/ Member address: gschwind@gnu-log.net
participants (2)
-
Benoit Gschwind -
Sebastian Berg