Hi all, I use the Numpy C API to produce/use some PyArrayObjects. To set the allocated memory zone, I use the PyArray_FromDimsAndData function, which is described to "be used to access global data that will never be freed". That's what I want. Or more exactly, I want "global data that will never be freed by Numpy, until I tell it to do so !" I mean some of my arrays are allocated and used as PyArrayObject data, but I want some of them to be seen by Numpy as its own data. I want it to deallocate the data at delete time. My questions are : [1] Is PyArray_FromDimsAndData the right function or should I use another way ? [2] Can I use the PyArrayObject.flags bit "owns the data area" to set it after the PyArray_FromDimsAndData call ? In the case of "yes", which is the bit rank ? The fourth starting from right ? Any macro already doing this ? Will I break the PyArrayObject consistency ? Marcvs [alias yes I can go into the code... I though OO was reading docs and using interfaces ;]
Marc Poinot <Marc.Poinot@onera.fr> writes:
That's what I want. Or more exactly, I want "global data that will never be freed by Numpy, until I tell it to do so !"
That option does not exist. The options are: 1) NumPy manages the data space of your array. It gets freed when the last array object referencing it is destroyed. 2) NumPy assumes that the data space is already allocated and is not freed as long as any array object might reference it (which, in practice, is until the end of the process). PyArray_FromDimsAndData is used for allocating arrays that choose the second option. If I understand you correctly, you want NumPy to create an array object and allocate the data space, but make sure that the data space is not freed before you "allow" it. In that case, just create an ordinary array and keep an additional reference to it. When the data space may be destroyed, you remove the reference. However, there is no guarantee that the data space will be freed immediately, as there might still be other references around.
[2] Can I use the PyArrayObject.flags bit "owns the data area" to set it after the PyArray_FromDimsAndData call ? In the case of "yes", which
Whatever this does, it is not documented. Konrad. -- ------------------------------------------------------------------------------- Konrad Hinsen | E-Mail: hinsen@cnrs-orleans.fr Centre de Biophysique Moleculaire (CNRS) | Tel.: +33-2.38.25.56.24 Rue Charles Sadron | Fax: +33-2.38.63.15.17 45071 Orleans Cedex 2 | Deutsch/Esperanto/English/ France | Nederlands/Francais -------------------------------------------------------------------------------
Konrad Hinsen wrote:
If I understand you correctly, you want NumPy to create an array object and allocate the data space, but make sure that the data space is not freed before you "allow" it. In that case, just create an ordinary array and keep an additional reference to it. When the data space may be destroyed, you remove the reference. However, there is no guarantee that the data space will be freed immediately, as there might still be other references around.
No. I want to set the memory zone of the array but once this zone is set, I want numpy to manage it as if it was owner of the memory. I have an external lib which allocates the returned memory zone. I put this memory zone into a PyArrayObject using PyArray_FromDimsAndData in order to avoid memory copy. But I want now this array to be the owner of the allocated zone. I mean I want this zone to be released if the Python object is deleted. The ref count of Python is ok for me, as long as an array is sharing the data, python won't release it. But I want Python to delete the memory zone if the last reference is removed. Marcvs [alias I'll have a try with myarray->flags |= OWN_DATA; and I'll let you know about my experiments...]
No. I want to set the memory zone of the array but once this zone is set, I want numpy to manage it as if it was owner of the memory.
That is the most frequent case for which there is no clean solution. There ought to be an array constructor that takes a pointer to a deallocation function which is called to free the data space. You can do myarray->flags |= OWN_DATA, then the data space will be freed using the standard free() function. But this is undocumented, and works only if the standard OS memory allocation calls were used to allocate the memory. Konrad. -- ------------------------------------------------------------------------------- Konrad Hinsen | E-Mail: hinsen@cnrs-orleans.fr Centre de Biophysique Moleculaire (CNRS) | Tel.: +33-2.38.25.56.24 Rue Charles Sadron | Fax: +33-2.38.63.15.17 45071 Orleans Cedex 2 | Deutsch/Esperanto/English/ France | Nederlands/Francais -------------------------------------------------------------------------------
participants (2)
-
Konrad Hinsen -
Marc Poinot