Determine if numpy is installed from an extension
Hello, I have written a C-extension for python that uses arrays from python, does calculations on them and returns a result on that. I have now also added the possibility to provide numpy arrays. However this is not a requirement. Python arrays (lists) are still allowed also. I check in the C-code which kind of arrays are provided. That all works ok, but I have one problem. In the initialisation function for the extension I call import_array to initialise the numpy library. The problem is if numpy is not installed on the system, that this call generates a message and an error and the rest of the routine is aborted. Via the source code of numpy I discovered that import_array is in fact a macro that calls _import_array and checks its return value to give the message. But even it I call _import_array myself and check for the return code, after the initialisation routine has finished I still get the message 'No module named numpy.core.multiarray'. How can I test if numpy is installed on the system from the extension so that I do not active the numpy functionality and that it is still able to use my extension, but then without numpy support? I have searched in the manual and documentation, searched in google, but I do not find an answer on that question.
Peter Notebaert wrote:
How can I test if numpy is installed on the system from the extension so that I do not active the numpy functionality and that it is still able to use my extension, but then without numpy support?
Is there some reason why you cannot try to import numpy first to check whether it is available ? cheers, David
From an extension? How to import numpy from there and then test if that succeeded and that without any annoying message if possible...
Thanks, Peter On Wed, Feb 3, 2010 at 1:34 AM, David Cournapeau <david@silveregg.co.jp>wrote:
Peter Notebaert wrote:
How can I test if numpy is installed on the system from the extension so that I do not active the numpy functionality and that it is still able to use my extension, but then without numpy support?
Is there some reason why you cannot try to import numpy first to check whether it is available ?
cheers,
David _______________________________________________ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
On Wed, Feb 3, 2010 at 5:38 PM, Peter Notebaert <peno@telenet.be> wrote:
From an extension? How to import numpy from there and then test if that succeeded and that without any annoying message if possible...
One obvious solution would be to simply call PyImport_Import, something like: #include <Python.h> PyMODINIT_FUNC initfoo(void) { PyObject *m, *mod; m = Py_InitModule("foo", NULL); if (m == NULL) { return; } mod = PyImport_ImportModule("numpy"); if (mod == NULL) { return; } Py_DECREF(mod); } But I am not sure whether it would cause some issues if you do this and then import the numpy C API (which is mandatory before using any C functions from numpy). I know the python import system has some dark areas, I don't know if that's one of them or not. cheers, David
On Wed, Feb 3, 2010 at 03:41, David Cournapeau <cournape@gmail.com> wrote:
On Wed, Feb 3, 2010 at 5:38 PM, Peter Notebaert <peno@telenet.be> wrote:
From an extension? How to import numpy from there and then test if that succeeded and that without any annoying message if possible...
One obvious solution would be to simply call PyImport_Import, something like:
#include <Python.h>
PyMODINIT_FUNC initfoo(void) { PyObject *m, *mod;
m = Py_InitModule("foo", NULL); if (m == NULL) { return; }
mod = PyImport_ImportModule("numpy"); if (mod == NULL) { return; } Py_DECREF(mod);
Or rather, to recover from the failed import as the OP wants to do: mod = PyImport_ImportModule("numpy"); if (mod == NULL) { /* Clear the error state since we are handling the error. */ PyErr_Clear(); /* ... set up for the sans-numpy case. */ } else { Py_DECREF(mod); import_array(); /* ... set up for the with-numpy case. */ } -- Robert Kern "I have come to believe that the whole world is an enigma, a harmless enigma that is made terrible by our own mad attempt to interpret it as though it had an underlying truth." -- Umberto Eco
Ah, that is maybe the idea: if (_import_array() < 0) { /* Clear the error state since we are handling the error. */ PyErr_Clear(); /* ... set up for the sans-numpy case. */ } else { /* ... set up for the with-numpy case. */ } I did not call PyErr_Clear() when _import_array() < 0 and the error is probably still hanging and then given later. I will try this this evening. Thank you for the hints. Peter On Wed, Feb 3, 2010 at 4:22 PM, Robert Kern <robert.kern@gmail.com> wrote:
On Wed, Feb 3, 2010 at 5:38 PM, Peter Notebaert <peno@telenet.be> wrote:
From an extension? How to import numpy from there and then test if that succeeded and that without any annoying message if possible...
One obvious solution would be to simply call PyImport_Import, something
On Wed, Feb 3, 2010 at 03:41, David Cournapeau <cournape@gmail.com> wrote: like:
#include <Python.h>
PyMODINIT_FUNC initfoo(void) { PyObject *m, *mod;
m = Py_InitModule("foo", NULL); if (m == NULL) { return; }
mod = PyImport_ImportModule("numpy"); if (mod == NULL) { return; } Py_DECREF(mod);
Or rather, to recover from the failed import as the OP wants to do:
mod = PyImport_ImportModule("numpy"); if (mod == NULL) { /* Clear the error state since we are handling the error. */ PyErr_Clear(); /* ... set up for the sans-numpy case. */ } else { Py_DECREF(mod); import_array(); /* ... set up for the with-numpy case. */ }
-- Robert Kern
"I have come to believe that the whole world is an enigma, a harmless enigma that is made terrible by our own mad attempt to interpret it as though it had an underlying truth." -- Umberto Eco _______________________________________________ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
Just to imform that my approach works: if (_import_array() < 0) { /* Clear the error state since we are handling the error. */ PyErr_Clear(); /* ... set up for the sans-numpy case. */ } else { /* ... set up for the with-numpy case. */ } It is based on Roberts idea to call PyImport_ImportModule("numpy"); and check if that succeededand clean up. In fact, _import_array() is doing this. The code of _import_array() is in the header file __multiarray_api.h in the numpy folder of the distributed files: static int _import_array(void) { int st; PyObject *numpy = PyImport_ImportModule("numpy.core.multiarray"); PyObject *c_api = NULL; if (numpy == NULL) return -1; c_api = PyObject_GetAttrString(numpy, "_ARRAY_API"); if (c_api == NULL) {Py_DECREF(numpy); return -1;} if (PyCObject_Check(c_api)) { PyArray_API = (void **)PyCObject_AsVoidPtr(c_api); } Py_DECREF(c_api); Py_DECREF(numpy); if (PyArray_API == NULL) return -1; /* Perform runtime check of C API version */ if (NPY_VERSION != PyArray_GetNDArrayCVersion()) { PyErr_Format(PyExc_RuntimeError, "module compiled against "\ "ABI version %x but this version of numpy is %x", \ (int) NPY_VERSION, (int) PyArray_GetNDArrayCVersion()); return -1; } if (NPY_FEATURE_VERSION > PyArray_GetNDArrayCFeatureVersion()) { PyErr_Format(PyExc_RuntimeError, "module compiled against "\ "API version %x but this version of numpy is %x", \ (int) NPY_FEATURE_VERSION, (int) PyArray_GetNDArrayCFeatureVersion()); return -1; } /* * Perform runtime check of endianness and check it matches the one set by * the headers (npy_endian.h) as a safeguard */ st = PyArray_GetEndianness(); if (st == NPY_CPU_UNKNOWN_ENDIAN) { PyErr_Format(PyExc_RuntimeError, "FATAL: module compiled as unknown endian"); return -1; } #if NPY_BYTE_ORDER ==NPY_BIG_ENDIAN if (st != NPY_CPU_BIG) { PyErr_Format(PyExc_RuntimeError, "FATAL: module compiled as "\ "big endian, but detected different endianness at runtime"); return -1; } #elif NPY_BYTE_ORDER == NPY_LITTLE_ENDIAN if (st != NPY_CPU_LITTLE) { PyErr_Format(PyExc_RuntimeError, "FATAL: module compiled as "\ "little endian, but detected different endianness at runtime"); return -1; } #endif return 0; } As you can see, this routine is doing the same at the beginning with additional tests and the return value indicates if ok or not. So I only had to call PyErr_Clear(); when it failes and the problem is solved. Thanks for your input. Peter From: Peter Notebaert Sent: Wednesday, February 03, 2010 16:57 To: Discussion of Numerical Python Subject: Re: [Numpy-discussion] Determine if numpy is installed from anextension Ah, that is maybe the idea: if (_import_array() < 0) { /* Clear the error state since we are handling the error. */ PyErr_Clear(); /* ... set up for the sans-numpy case. */ } else { /* ... set up for the with-numpy case. */ } I did not call PyErr_Clear() when _import_array() < 0 and the error is probably still hanging and then given later. I will try this this evening. Thank you for the hints. Peter On Wed, Feb 3, 2010 at 4:22 PM, Robert Kern <robert.kern@gmail.com> wrote: On Wed, Feb 3, 2010 at 03:41, David Cournapeau <cournape@gmail.com> wrote:
On Wed, Feb 3, 2010 at 5:38 PM, Peter Notebaert <peno@telenet.be> wrote:
From an extension? How to import numpy from there and then test if that succeeded and that without any annoying message if possible...
One obvious solution would be to simply call PyImport_Import, something like:
#include <Python.h>
PyMODINIT_FUNC initfoo(void) { PyObject *m, *mod;
m = Py_InitModule("foo", NULL); if (m == NULL) { return; }
mod = PyImport_ImportModule("numpy"); if (mod == NULL) { return; } Py_DECREF(mod);
Or rather, to recover from the failed import as the OP wants to do: mod = PyImport_ImportModule("numpy"); if (mod == NULL) { /* Clear the error state since we are handling the error. */ PyErr_Clear(); /* ... set up for the sans-numpy case. */ } else { Py_DECREF(mod); import_array(); /* ... set up for the with-numpy case. */ } -- Robert Kern "I have come to believe that the whole world is an enigma, a harmless enigma that is made terrible by our own mad attempt to interpret it as though it had an underlying truth." -- Umberto Eco _______________________________________________ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion -------------------------------------------------------------------------------- _______________________________________________ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
participants (4)
-
David Cournapeau -
David Cournapeau -
Peter Notebaert -
Robert Kern