Creating PyArrayObject in C++
I've built an analysis framework for studying electrophysiogical data, and I'd like to add NumPy (and SciPy, although that's not germane to my current question) as an engine so users can more easily add their own functions to the program. I'm at the point now of trying to share the data between the C++ program and the python interpreter. Keep in mind, these can be large datasets (some of our data is sampled at 40,000 Hz), so I'd like to minimize copying data as much as possible. My first solution is to use PyArray_FromDimsAndData(). I know that it's not recommended, but the data is owned by an object that takes care of freeing it, so it would look like the following: class Unit() { public: Unit(int size, int *data) : data_(data), pyArray_(0) { pyArray_ = PyArrayFromDimsAndData(1, &size, PyArray_INT, (char*)data_); } } ~Unit() { Py_DECREF(pyArray_); delete[] data_; } private: int *data_; PyArrayObject* pyArray_; }; I can guarantee that all analysis will be finished before the Unit object is destroyed and delete[] is called, so I don't think that's a problem. However, what if there are modifications to the array from within a Python script (specifically changes to the number of elements in the array). I should be able to get the new size from PyArrayObject, but will it have any effect on the delete[] operation? Is there a chance that PyArrayObject.data could change from within a Python script, necessitating code such as: if (data_ == pyArray_.data) { delete[] data_; } else { delete[] data_; delete[] pyArray_.data); } Or is there a better way to do this without expensive copy operations that I haven't come across yet? Thanks, Kenny
Hi Kenny
2009/7/7 Kenny Abernathy
I can guarantee that all analysis will be finished before the Unit object is destroyed and delete[] is called, so I don't think that's a problem.
There is a neat trick to make sure things don't get deallocated out of order: http://blog.enthought.com/?p=62
However, what if there are modifications to the array from within a Python script (specifically changes to the number of elements in the array). I should be able to get the new size from PyArrayObject, but will it have any effect on the delete[] operation?
NumPy won't reallocate any of your memory if you set the array flag OWNDATA to False, so it should be safe to deallocate as usual. Regards Stéfan
2009/7/7 Stéfan van der Walt
Hi Kenny
2009/7/7 Kenny Abernathy
: I can guarantee that all analysis will be finished before the Unit object is destroyed and delete[] is called, so I don't think that's a problem.
There is a neat trick to make sure things don't get deallocated out of order:
Moreover, if you use PyCObject_FromVoidPtr() http://docs.python.org/c-api/cobject.html, you can register a deallocation routine to be called at DECREF time, so no need at all of implementing your a brand new type. -- Lisandro Dalcín --------------- Centro Internacional de Métodos Computacionales en Ingeniería (CIMEC) Instituto de Desarrollo Tecnológico para la Industria Química (INTEC) Consejo Nacional de Investigaciones Científicas y Técnicas (CONICET) PTLC - Güemes 3450, (3000) Santa Fe, Argentina Tel/Fax: +54-(0)342-451.1594
Thanks, you two. That helps alot. The PyCObject_FromVoidPtr() trick is
good to know--I still have to have my class because it holds other data, but
this definitely points me in the right direction.
On Tue, Jul 7, 2009 at 9:55 PM, Lisandro Dalcin
2009/7/7 Stéfan van der Walt
: Hi Kenny
2009/7/7 Kenny Abernathy
: I can guarantee that all analysis will be finished before the Unit object is destroyed and delete[] is called, so I don't think that's a problem.
There is a neat trick to make sure things don't get deallocated out of order:
Moreover, if you use PyCObject_FromVoidPtr() http://docs.python.org/c-api/cobject.html, you can register a deallocation routine to be called at DECREF time, so no need at all of implementing your a brand new type.
-- Lisandro Dalcín --------------- Centro Internacional de Métodos Computacionales en Ingeniería (CIMEC) Instituto de Desarrollo Tecnológico para la Industria Química (INTEC) Consejo Nacional de Investigaciones Científicas y Técnicas (CONICET) PTLC - Güemes 3450, (3000) Santa Fe, Argentina Tel/Fax: +54-(0)342-451.1594 _______________________________________________ Numpy-discussion mailing list Numpy-discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
participants (3)
-
Kenny Abernathy
-
Lisandro Dalcin
-
Stéfan van der Walt