[C++-sig] create boost array from a Numpy PyArray without copying data?

Sebastian Walter walter at mathematik.hu-berlin.de
Thu Jan 22 11:51:20 CET 2009


-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Ravi schrieb:
> On Wednesday 21 January 2009 03:40:00 Sebastian Walter wrote:
>> Apparently I have to increase the refcount manually when I return a raw
>> PyObject. So I wrote getter functions like that:
>>
>>         PyObject* get_rhs(){
>>                 return incref((PyObject *) _array_my_rhs);
>>         }
>> I'm not sure that this is the way it should be done. But it solved the
>> memory corruption problem.
> 
> In this case, you are essentially creating a memory leak. The returned array 
> will never be deleted by python since its reference count will never go to 
> zero.
I'm sorry, the above code is not really self contained and now I'm not
sure that we are talking about the same thing. I do not intend to EVER
free the memory of *x!! So I don't have to worry about
“make sure the memory doesn’t get deallocated before the NumPy
array disappears.”
because *x is global!

So here is the code that I actually wanted to post:

- ------------------ test.cpp -------------------
#define PY_ARRAY_UNIQUE_SYMBOL PyArrayHandle
#include <iostream>
#include <boost/python.hpp>
#include <numpy/noprefix.h>

using namespace std;
using namespace boost;
using namespace boost::python;
using namespace boost::python::numeric;

double x[] = {1.,2.,3.};
int N = 3;

class A{
	public:
	PyArrayObject *my_array;

	A(){
		my_array  = (PyArrayObject*)PyArray_SimpleNewFromData(1, &N,
PyArray_DOUBLE, x);
	}

	PyObject* get_my_array(){
		return incref((PyObject*) my_array);
	}
};



BOOST_PYTHON_MODULE(_test)
{
	import_array();
	array::set_module_and_type("numpy", "ndarray");

	class_<A>("A", init<>())
	.add_property("my_array", &A::get_my_array)
	;
}

- --------------- end test.cpp ---------------------

- --------------- test.py ------------------
from _test import *

def test_my_array():
	a = A()
	print a.my_array
	print a.my_array
- ---------------- end test.py ----------------


If I don't put the incref I get an
*** glibc detected *** python: malloc(): memory corruption (fast):
0x0817b6a1 ***



> Please see http://blog.enthought.com/?p=62 for information on fixing 
> this issue. The code I pointed out earlier encapsulates this using a custom 
> deleter in C++ which can be used independently of ublas.

I cite from that blog:
"""The suggestions have always been something similar to “make sure the
memory doesn’t get deallocated before the NumPy
array disappears.” This is nice advice, but not generally helpful as it
basically just tells you to create a memory leak.

All that NumPy does internally is to un-set a flag on the array object
indicating that the array doesn’t own its memory pointer and so NumPy
won’t free the memory when the last reference to the array disappears."""

So that means:
In my above case I know that *x won't be deleted because it is global,
right?
Why exactly does the
*** glibc detected *** python: malloc(): memory corruption (fast):
0x0817b6a1 ***
occur then?


And is there a way to get the refcount of an object within Python? I'd
like to test if the refcount is correct.



regards,
Sebastian
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v2.0.4-svn0 (GNU/Linux)
Comment: Using GnuPG with SUSE - http://enigmail.mozdev.org

iD8DBQFJeE+o9PBA5IG0h0ARAjeQAKCR+HQQ2XjiqLAO01ZwWRqIC5KUcgCgl+ne
b80vmGWmi6jLLuxeOyjOt0w=
=mmAr
-----END PGP SIGNATURE-----


More information about the Cplusplus-sig mailing list