Please help - pointer to slice
Hi, I am using SWIG to wrap some C++ function operating on numarray arrays. Everything works quite well -- up until yesterday... I have some 3d data and was thinking if the actual operation I want to do is done "slice by slice", I could just loop over Z in python and have my C function defined like this: void DoSomething(float * array2d, int nx, int ny) The python loop looks like this: arr = numarray.array(shape=(100,100,100), type=numarray.Float32) for z in range(nz): DoSomething( arr[z] ) SOMEHOW the pointer that get transfered to the C++ side is always the same as for the full (3d) array. if I try to debug this by doing: for z in range(nz): print repr(arr[z]._data) that also tells be that python / numarray thinks all slices are located at the same address, that is: every slice looks just like the full array. Please help, I don't know what to do... Sebastian Haase
A Divendres 07 Març 2003 01:42, Sebastian Haase va escriure:
if I try to debug this by doing: for z in range(nz): print repr(arr[z]._data)
that also tells be that python / numarray thinks all slices are located at the same address, that is: every slice looks just like the full array.
You can access the slices by taking into account the byte offset
attribute. Look at the next example:
In [20]: a=numarray.arange(100, shape=(10,10))
In [21]: a._data
Out[21]:
Thanks, SO MUCH ...
I almost went crazy yesterday - and was really desperate by the time I wrote
that email.
Somehow I newer needed that offset field until now. So: when I do the
NA_InputArray call I get a "proper" C-array
just that it does NOT necessarily start at NAimg->data
but rather at
NAimg->data + NAimg->byteoffset
Again: thanks so much.
BTW: Is there general interest in my SWIG typemaps. (SWIG is maybe the
easiest way to wrap C/C++ functions (and classes)
into Python (and/or Perl, Java, Ruby,...) ? I think especially for
numerical stuff, that "link" in of interest.
Sebastian
----- Original Message -----
From: "Francesc Alted"
if I try to debug this by doing: for z in range(nz): print repr(arr[z]._data)
that also tells be that python / numarray thinks all slices are located at the same address, that is: every slice looks just like the full array.
You can access the slices by taking into account the byte offset
attribute. Look at the next example:
In [20]: a=numarray.arange(100, shape=(10,10))
In [21]: a._data
Out[21]:
Sebastian Haase wrote:
Thanks, SO MUCH ... I almost went crazy yesterday - and was really desperate by the time I wrote that email. Somehow I newer needed that offset field until now. So: when I do the NA_InputArray call I get a "proper" C-array just that it does NOT necessarily start at NAimg->data but rather at NAimg->data + NAimg->byteoffset
Well, in case you (or others) find it useful, I'm including here a little
library I wrote for accessing general Numeric 2-d arrays (contiguous or not)
in an easy manner.
Here's a snippet of a simple (included) example of a function to print an
integer array:
static PyObject *idisp(PyObject *self, PyObject *args)
{
PyArrayObject *array;
int **arr; // for the data area
int i,j,cs;
if (!PyArg_ParseTuple(args, "O!",&PyArray_Type,&array ) )
return NULL;
arr = imatrix_data(array,&cs);
for (i=0;i<array->dimensions[0];++i) {
for (j=0;j
Again: thanks so much. BTW: Is there general interest in my SWIG typemaps. (SWIG is maybe the easiest way to wrap C/C++ functions (and classes) into Python (and/or Perl, Java, Ruby,...) ? I think especially for numerical stuff, that "link" in of interest.
I'd love to see them. So far I've either used high-level stuff like weave.inline() or just written the extensions by hand (as in the code I'm supplying here). I'd like to see this, especially if there is an easy way to handle contiguity issues with it. Best, f.
Well, in case you (or others) find it useful, I'm including here a little library I wrote for accessing general Numeric 2-d arrays (contiguous or not) in an easy manner.
Here's a snippet of a simple (included) example of a function to print an integer array:
static PyObject *idisp(PyObject *self, PyObject *args) { PyArrayObject *array; int **arr; // for the data area int i,j,cs;
if (!PyArg_ParseTuple(args, "O!",&PyArray_Type,&array ) ) return NULL;
arr = imatrix_data(array,&cs); for (i=0;i<array->dimensions[0];++i) { for (j=0;j
dimensions[1];j+=cs) printf("%5d ",arr[i][j]); printf("\n"); } free(arr); Py_INCREF(Py_None); return Py_None; }
You get the **arr pointer and you can then manipulate it as a[i][j] conveniently. The supplied example file may be enough for many to write
<snip> their
Numpy C extensions without much trouble.
Again: thanks so much. BTW: Is there general interest in my SWIG typemaps. (SWIG is maybe the easiest way to wrap C/C++ functions (and classes) into Python (and/or Perl, Java, Ruby,...) ? I think especially for numerical stuff, that "link" in of interest.
I'd love to see them. So far I've either used high-level stuff like weave.inline() or just written the extensions by hand (as in the code I'm supplying here). I'd like to see this, especially if there is an easy way to handle contiguity issues with it.
You are using Numeric not numarray , right?
In any case :
My thinking goes as follows:
I was looking for a wrapping-solution that would be most transparent to the
people that would write the C/C++ code (or even Fortran, BTW)
Since I already had some experience using SWIG, that's what I wanted to use
to handle the "numarray binding".
SWIG has a quite strong (meaning: flexible, general) way of handling data
types. They call it "type maps":
Once you have that in place (later more) the whole "binding" looks like
this: (an example)
C side:
double sebFunc(float *arr, int nx, int ny, int nz) { double a =0; for(int
i=0;i
Sebastian Haase wrote:
Thanks, SO MUCH ... I almost went crazy yesterday - and was really desperate by the time I wrote that email. Somehow I newer needed that offset field until now. So: when I do the NA_InputArray call I get a "proper" C-array just that it does NOT necessarily start at NAimg->data but rather at NAimg->data + NAimg->byteoffset
The numarray-0.4 manual (available at: http://prdownloads.sourceforge.net/numpy/numarray-0.4.pdf?download) documents how to write Python stubs using the "high level" API on Page 70. The thing you appear to be missing is a call to NA_OFFSETDATA which gets a pointer to the data in an array by adding the buffer pointer and byteoffset together. Todd
Again: thanks so much. BTW: Is there general interest in my SWIG typemaps. (SWIG is maybe the easiest way to wrap C/C++ functions (and classes) into Python (and/or Perl, Java, Ruby,...) ? I think especially for numerical stuff, that "link" in of interest.
Sebastian
----- Original Message ----- From: "Francesc Alted"
To: "Sebastian Haase" ; Sent: Thursday, March 06, 2003 9:40 PM Subject: Re: [Numpy-discussion] Please help - pointer to slice A Divendres 07 Març 2003 01:42, Sebastian Haase va escriure:
if I try to debug this by doing: for z in range(nz): print repr(arr[z]._data)
that also tells be that python / numarray thinks all slices are located at the same address, that is: every slice looks just like the full array.
You can access the slices by taking into account the byte offset attribute. Look at the next example:
In [20]: a=numarray.arange(100, shape=(10,10))
In [21]: a._data Out[21]:
In [22]: a[1]._data Out[22]:
as you already know, both memory buffers point to the same address. I guess this is implemented in that way so as to not copy data unnecessarily.
now, look at:
In [23]: a._byteoffset Out[23]: 0
In [24]: a[1]._byteoffset Out[24]: 40
So, you can know where the data actually starts by looking at the _byteoffset property. In fact, you should always look at it in order to get proper results!.
In C, you can access to this information by looking at the byteoffset field in the PyArrayObject struct (look at chapter 10 in the User's Manual).
Hope that helps,
-- Francesc Alted
------------------------------------------------------- This SF.net email is sponsored by: Etnus, makers of TotalView, The debugger for complex code. Debugging C/C++ programs can leave you feeling lost and disoriented. TotalView can help you find your way. Available on major UNIX and Linux platforms. Try it free. www.etnus.com _______________________________________________ Numpy-discussion mailing list Numpy-discussion@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/numpy-discussion
participants (4)
-
Fernando Perez
-
Francesc Alted
-
Sebastian Haase
-
Todd Miller