Best/Safest way to parse NumPy array in C extension?
I've done this two ways. Which is better or safer? Say I'm passing a NumPy array (float) to a C extension. One way to "parse" it is (ignoring checks on return values, etc. for now), PyArrayObject *mat; PyArg_ParseTuple(args, "O!", &PyArray_Type, &mat); or is this better, PyObject *Pymat; PyArrayObject *mat; PyArg_ParseTuple(args, "O", &Pymat); mat=PyArray_ContiguousFromObject(objin,NPY_DOUBLE, 2,2); The latter appears to have the constraint that the array be contiguous. Or is that an illusion and it's saying it _expects_ a Python object that has contiguous data? I've done both. Pointing C arrays to the resulting PyArrays' data works fine, but I fear one way or the other might be courting disaster. BTW, I do other checks on dimension, type, etc., but I left those out here for clarity. (I hope.) -- Lou Pecora My views are my own. ____________________________________________________________________________________ Do you Yahoo!? Everyone is raving about the all-new Yahoo! Mail beta. http://new.mail.yahoo.com
YIKES. That one line should read mat=PyArray_ContiguousFromObject(Pymat,NPY_DOUBLE,2,2); Very sorry. --- Lou Pecora <lou_boog2000@yahoo.com> wrote:
I've done this two ways. Which is better or safer?
Say I'm passing a NumPy array (float) to a C extension. One way to "parse" it is (ignoring checks on return values, etc. for now),
PyArrayObject *mat; PyArg_ParseTuple(args, "O!", &PyArray_Type, &mat);
or is this better,
PyObject *Pymat; PyArrayObject *mat; PyArg_ParseTuple(args, "O", &Pymat); mat=PyArray_ContiguousFromObject(objin,NPY_DOUBLE, 2,2);
The latter appears to have the constraint that the array be contiguous. Or is that an illusion and it's saying it _expects_ a Python object that has contiguous data?
I've done both. Pointing C arrays to the resulting PyArrays' data works fine, but I fear one way or the other might be courting disaster.
BTW, I do other checks on dimension, type, etc., but I left those out here for clarity. (I hope.)
-- Lou Pecora My views are my own.
____________________________________________________________________________________
Do you Yahoo!? Everyone is raving about the all-new Yahoo! Mail beta. http://new.mail.yahoo.com _______________________________________________ SciPy-user mailing list SciPy-user@scipy.org
http://projects.scipy.org/mailman/listinfo/scipy-user
-- Lou Pecora My views are my own. ____________________________________________________________________________________ Have a burning question? Go to www.Answers.yahoo.com and get answers from real people who know.
Lou Pecora wrote:
I've done this two ways. Which is better or safer?
Say I'm passing a NumPy array (float) to a C extension. One way to "parse" it is (ignoring checks on return values, etc. for now),
PyArrayObject *mat; PyArg_ParseTuple(args, "O!", &PyArray_Type, &mat);
or is this better,
PyObject *Pymat; PyArrayObject *mat; PyArg_ParseTuple(args, "O", &Pymat); mat=PyArray_ContiguousFromObject(objin,NPY_DOUBLE, 2,2);
It totally depends on what you want. If you just pass in the array, then you will have to make sure that your code can handle the full generality of the ndarray (or else have checks on the Python-side). Ndarrays can be arbitrarily strided, mis-aligned, out of machine order, and of arbitrary data-type.
The latter appears to have the constraint that the array be contiguous. Or is that an illusion and it's saying it _expects_ a Python object that has contiguous data?
No, the latter will convert an array to be contiguous, in machine byteorder (and aligned) and convert it to DOUBLE type. So, if your code only handles that kind of array, then use the conversion function.
I've done both. Pointing C arrays to the resulting PyArrays' data works fine, but I fear one way or the other might be courting disaster.
It will be fine until you give it a discontiguous array, in the wrong byte-order, and possibly mis-aligned (coming from a field of another array). -Travis
Thanks for the feedback, Travis. Chris Barker gave some similar answers, but you both provided non-overlapping replies. A few questions: --- Travis Oliphant <oliphant.travis@ieee.org> wrote:
--- Lou Pecora wrote:
or is this better,
PyObject *Pymat; PyArrayObject *mat; PyArg_ParseTuple(args, "O", &Pymat);
mat=PyArray_ContiguousFromObject(objin,NPY_DOUBLE,
2,2);
It totally depends on what you want. If you just pass in the array, then you will have to make sure that your code can handle the full generality of the ndarray (or else have checks on the Python-side).
I will have to learn how to check for contiguous in Python.
Ndarrays can be arbitrarily strided, mis-aligned, out of machine order, and of arbitrary data-type.
The latter appears to have the constraint that the array be contiguous. Or is that an illusion and it's saying it _expects_ a Python object that has contiguous data?
No, the latter will convert an array to be contiguous, in machine byteorder (and aligned) and convert it to DOUBLE type. So, if your code only handles that kind of array, then use the conversion function.
Does this conversion mean that the array now has new data in contiguous form so that operations done on the data in the C extension (in place) will show up in that array when I return to the Python caller? (I am assuming I passed the array in as an arg to the C extension in the first place.)
It will be fine until you give it a discontiguous array, in the wrong byte-order, and possibly mis-aligned (coming from a field of another array).
-Travis
Got it. It seems that NumPy array creators like zeros(...) return continguous data. Is that always true? -- Lou Pecora My views are my own. ____________________________________________________________________________________ Need a quick answer? Get one in minutes from people who know. Ask your question on www.Answers.yahoo.com
On 12/12/06, Lou Pecora <lou_boog2000@yahoo.com> wrote:
I will have to learn how to check for contiguous in Python.
Easy: In [1]: import numpy as N In [2]: a=N.arange(10) In [3]: b=a[::2] In [4]: a.flags.contiguous Out[4]: True In [5]: b.flags.contiguous Out[5]: False Cheers, f
Thanks. I also found .flags.contiguous after some googling. You give a good example for the non-contiguous case. -- Lou --- Fernando Perez <fperez.net@gmail.com> wrote:
On 12/12/06, Lou Pecora <lou_boog2000@yahoo.com> wrote:
I will have to learn how to check for contiguous in Python.
Easy:
In [1]: import numpy as N
In [2]: a=N.arange(10)
In [3]: b=a[::2]
In [4]: a.flags.contiguous Out[4]: True
In [5]: b.flags.contiguous Out[5]: False
Cheers,
f
____________________________________________________________________________________ Do you Yahoo!? Everyone is raving about the all-new Yahoo! Mail beta. http://new.mail.yahoo.com
I have some notes and code about C extensions on the SciPy.org Wiki. Now I have upgraded the attachments I put there and would like to remove the old files. I cannot find how to do this either by searching SciPy site or googling. Anyone know? -- Lou Pecora My views are my own. ____________________________________________________________________________________ Do you Yahoo!? Everyone is raving about the all-new Yahoo! Mail beta. http://new.mail.yahoo.com
Lou Pecora wrote:
I have some notes and code about C extensions on the SciPy.org Wiki. Now I have upgraded the attachments I put there and would like to remove the old files. I cannot find how to do this either by searching SciPy site or googling. Anyone know?
Simply ask to add you to the Editors List page. -- http://scipy.org/FredericPetit
Uh... ask whom? --- fred <fredmfp@gmail.com> wrote:
Simply ask to add you to the Editors List page.
-- Lou Pecora My views are my own. ____________________________________________________________________________________ Cheap talk? Check out Yahoo! Messenger's low PC-to-Phone call rates. http://voice.yahoo.com
Lou Pecora wrote:
Uh... ask whom?
Hmm, ask here ;-) I and Gaël have just been added to the list last week (by R. Kern IIRC). Browse archive mailing-list ;-) -- http://scipy.org/FredericPetit
fred wrote:
Lou Pecora wrote:
Uh... ask whom?
Hmm, ask here ;-)
I and Gaël have just been added to the list last week (by R. Kern IIRC).
And in fact, you could have him yourself. Anyways, Lou, I have now added you to http://www.scipy.org/EditorsGroup -- Robert Kern "I have come to believe that the whole world is an enigma, a harmless enigma that is made terrible by our own mad attempt to interpret it as though it had an underlying truth." -- Umberto Eco
Thanks, Robert. I will see what powers that gives me. :-) Deleting those files would be fine for starters. --- Robert Kern <robert.kern@gmail.com> wrote:
And in fact, you could have him yourself. Anyways, Lou, I have now added you to
http://www.scipy.org/EditorsGroup -- Robert Kern
-- Lou Pecora My views are my own. ____________________________________________________________________________________ Do you Yahoo!? Everyone is raving about the all-new Yahoo! Mail beta. http://new.mail.yahoo.com
participants (5)
-
Fernando Perez -
fred -
Lou Pecora -
Robert Kern -
Travis Oliphant