Hi! I need to wrap a pre-existent C array into a numarray. It shouldn't be a copy of the original array because this is an audio buffer with a fixed address and also input/output copying from/to the buffer would be overkilling, so the numarray *must* be lightly wrapping the buffer and not a memcpy of it. But NA_NewArray duplicates the passed buffer so I'm not sure how to achieve the goal stated above or if it's possible at all. Can you help me? Regards, Carlos
On Tue, May 17, 2005 at 09:36:22PM -0300, Carlos Pita wrote:
Hi! I need to wrap a pre-existent C array into a numarray. It shouldn't be a copy of the original array because this is an audio buffer with a fixed address and also input/output copying from/to the buffer would be overkilling, so the numarray *must* be lightly wrapping the buffer and not a memcpy of it. But NA_NewArray duplicates the passed buffer so I'm not sure how to achieve the goal stated above or if it's possible at all. Can you help me?
I'm not sure for Numarray, but for Numeric, you can use PyArray_FromDimsAndData(). There should be a similar function in numarray. -- Alexandre Fayolle LOGILAB, Paris (France). http://www.logilab.com http://www.logilab.fr http://www.logilab.org
On Wed, 2005-05-18 at 04:01, Alexandre wrote:
On Tue, May 17, 2005 at 09:36:22PM -0300, Carlos Pita wrote:
Hi! I need to wrap a pre-existent C array into a numarray. It shouldn't be a copy of the original array because this is an audio buffer with a fixed address and also input/output copying from/to the buffer would be overkilling, so the numarray *must* be lightly wrapping the buffer and not a memcpy of it. But NA_NewArray duplicates the passed buffer so I'm not sure how to achieve the goal stated above or if it's possible at all. Can you help me?
I'm not sure for Numarray, but for Numeric, you can use PyArray_FromDimsAndData(). There should be a similar function in numarray.
That function does indeed exist for Numeric and has non-copying semantics. Regards, Todd
The documentation for PyArray_FromDimsAndData that I quoted in my previous post seems to suggest that a copy is done. But browsing the web numarray cvs I find out that its implementation first creates a PyBuffer from the passed data: buf = PyBuffer_FromReadWriteMemory(data, nelements * descr->elsize); a = NA_NewAllFromBuffer( nd, dimensions, descr->type_num, buf, 0, descr->elsize, NA_ByteOrder(), 1, 1); Then, from the NA_NewAllFromBuffer doc (in numarray manual): It is possible to create a Python buffer object from an array of C data and then construct a numarray using this function which refers to the C data without making a copy. So you both are right. I guess creating a PyBuffer as above and then calling NA_NewAllFromBuffer is the best way to do what I was asking for in numarray (that is, without recurring to the numeric simulation API). Am I right? Regards, Carlos
On Wed, 2005-05-18 at 04:01, Alexandre wrote:
On Tue, May 17, 2005 at 09:36:22PM -0300, Carlos Pita wrote:
Hi! I need to wrap a pre-existent C array into a numarray. It shouldn't be a copy of the original array because this is an audio buffer with a fixed address and also input/output copying from/to the buffer would be overkilling, so the numarray *must* be lightly wrapping the buffer and not a memcpy of it. But NA_NewArray duplicates the passed buffer so I'm not sure how to achieve the goal stated above or if it's possible at all. Can you help me?
I'm not sure for Numarray, but for Numeric, you can use PyArray_FromDimsAndData(). There should be a similar function in numarray.
That function does indeed exist for Numeric and has non-copying semantics.
Regards, Todd
------------------------------------------------------- This SF.Net email is sponsored by Oracle Space Sweepstakes Want to be the first software developer in space? Enter now for the Oracle Space Sweepstakes! http://ads.osdn.com/?ad_id=7412&alloc_id=16344&op=click _______________________________________________ Numpy-discussion mailing list Numpy-discussion@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/numpy-discussion
On Wed, 2005-05-18 at 12:18, Carlos Pita wrote:
The documentation for PyArray_FromDimsAndData that I quoted in my previous post seems to suggest that a copy is done. But browsing the web
I'd appreciate it if you could send me a reference to where that is because it is stale.
numarray cvs I find out that its implementation first creates a PyBuffer from the passed data:
buf = PyBuffer_FromReadWriteMemory(data, nelements * descr->elsize); a = NA_NewAllFromBuffer( nd, dimensions, descr->type_num, buf, 0, descr->elsize, NA_ByteOrder(), 1, 1);
Then, from the NA_NewAllFromBuffer doc (in numarray manual):
It is possible to create a Python buffer object from an array of C data and then construct a numarray using this function which refers to the C data without making a copy.
So you both are right. I guess creating a PyBuffer as above and then calling NA_NewAllFromBuffer is the best way to do what I was asking for in numarray (that is, without recurring to the numeric simulation API).
Am I right?
Not in my opinion, for two reasons: 1. PyArray_FromDimsAndData is easier to use, internalizing the buffer creation step. 2. These days, my recommendation is to use the Numeric compatibility layer because it gives the best portability, to Numeric now and to Numeric3 in the future. Most of the NA_ routines exist to facilitate the implementation of numarray itself and because we didn't always know a compatibility layer was possible or desirable. Of course you're free to use the NA_ routines and we're not planning on removing them, but I think the Numeric compatible routines are a better choice. Regards, Todd
The documentation for PyArray_FromDimsAndData that I quoted in my previous post seems to suggest that a copy is done. But browsing the web
I'd appreciate it if you could send me a reference to where that is because it is stale.
I take the excerpt from the numarray user manual from the python2.3-numarray 1.1.1 debian (sarge) package. But it's updated in the cvs: \begin{cfuncdesc}{PyObject*}{PyArray_FromDimsAndData}{int nd, int *dims, int type, char *data} This function will allocate a numarray of the specified shape and type which will refer to the data buffer specified by \var{data}. The contents of \var{data} will not be copied nor will \var{data} be deallocated upon the deletion of the array. \end{cfuncdesc} It seems to be ok. When it says "nor will \var{data} be deallocated upon the deletion of the array", does it mean that the python buffer object ref count won't be decremented ? Or, more generally, the created numarray will refer directly to the C array segments exposed by the python buffer object or will keep a reference to this object?
Not in my opinion, for two reasons:
1. PyArray_FromDimsAndData is easier to use, internalizing the buffer creation step.
2. These days, my recommendation is to use the Numeric compatibility layer because it gives the best portability, to Numeric now and to
I will take your advice. Thank you! Regards, Carlos
On Wed, 2005-05-18 at 09:03, Todd Miller wrote:
On Wed, 2005-05-18 at 04:01, Alexandre wrote:
On Tue, May 17, 2005 at 09:36:22PM -0300, Carlos Pita wrote:
Hi! I need to wrap a pre-existent C array into a numarray. It shouldn't be a copy of the original array because this is an audio buffer with a fixed address and also input/output copying from/to the buffer would be overkilling, so the numarray *must* be lightly wrapping the buffer and not a memcpy of it. But NA_NewArray duplicates the passed buffer so I'm not sure how to achieve the goal stated above or if it's possible at all. Can you help me?
I'm not sure for Numarray, but for Numeric, you can use PyArray_FromDimsAndData(). There should be a similar function in numarray.
That function does indeed exist for Numeric and has non-copying semantics.
I misspoke here, meaning to say numarray. More in a subsequent post. Todd
But as NA_NewArray does, FromDimsAndData copies its contents from the passed pointer, which is what I'm willing to avoid. PyObject* PyArray FromDimsAndData(int nd, int *dims, int type, char *data) This function will allocate a numarray of the specified shape and type, and it's contents will be copied from the specified data pointer. The fact is that the audio buffer to be wrapped is a simple, fixed-size and contiguous C array, so I guess it should be easy to access through a numarray. But NA_NewArray and friends all seems to copy contents to a buffer of their own. Thank you. Regards, Carlos
On Tue, May 17, 2005 at 09:36:22PM -0300, Carlos Pita wrote:
Hi! I need to wrap a pre-existent C array into a numarray. It shouldn't be a copy of the original array because this is an audio buffer with a fixed address and also input/output copying from/to the buffer would be overkilling, so the numarray *must* be lightly wrapping the buffer and not a memcpy of it. But NA_NewArray duplicates the passed buffer so I'm not sure how to achieve the goal stated above or if it's possible at all. Can you help me?
I'm not sure for Numarray, but for Numeric, you can use PyArray_FromDimsAndData(). There should be a similar function in numarray.
participants (3)
-
Alexandre -
Carlos Pita -
Todd Miller