(newbie) How can I use NumPy to wrap my C++ class with 2dimensional arrays?
Hi everyone, (I sent this message yesterday as well but somehow it didn't come through...) I would like to ask your advice how I can use NumPy to wrap my existing C++ library with 2dimensional arrays. I am wrapping the library with swig and I have the typemap function declaration. But now I am struggling a lot to find an implementation for getting access to the 2dimensional arrays in my C++ classes. I have wrapped most of the library but the multidimensional arrays are problematic. From what I have read, I can use NumPy for this.. There is so much information about NumPy, Numeric etc that it is not clear anymore which example I can use best. It's clear that NumPy is probably the way to go. This is a small example of the class that I would like to wrap: struct MyClass { float array1[2][2]; std::vector<float> array2[2]; }; I've also read that I need to call import_array() but where should I put this? Is there a small example with NumPy capi to wrap/convert my 2dimensional arrays to python? Thanks a lot already Raymond
Hi,
In fact, it's not that complicated. You may know the way how copying a
vector, and this is all you need
(http://matt.eifelle.com/2008/01/04/transformingacvectorintoanumpyarra...).
You will have to copy your data, it is the safest way to ensure that
the data is always valid.
For the std::vector<>[], I suggest you convert it to a single vector,
as the data inside this "array" is not contiguous and it can thus be
cumbersome to create a Numpy array from that.
Once the structure is prepared, you have to allocate the dimensions,
but this may be available online.
In case you know that the C++ data won't go away before the Python
array, you can always wrap the container
(http://matt.eifelle.com/2008/11/04/exposinganarrayinterfacewithswigfor...)
with SWIG.
Matthieu
2009/7/30 Raymond de Vries
Hi everyone,
(I sent this message yesterday as well but somehow it didn't come through...)
I would like to ask your advice how I can use NumPy to wrap my existing C++ library with 2dimensional arrays. I am wrapping the library with swig and I have the typemap function declaration. But now I am struggling a lot to find an implementation for getting access to the 2dimensional arrays in my C++ classes. I have wrapped most of the library but the multidimensional arrays are problematic. From what I have read, I can use NumPy for this..
There is so much information about NumPy, Numeric etc that it is not clear anymore which example I can use best. It's clear that NumPy is probably the way to go.
This is a small example of the class that I would like to wrap: struct MyClass { float array1[2][2]; std::vector<float> array2[2]; };
I've also read that I need to call import_array() but where should I put this?
Is there a small example with NumPy capi to wrap/convert my 2dimensional arrays to python?
Thanks a lot already Raymond _______________________________________________ NumPyDiscussion mailing list NumPyDiscussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpydiscussion
 Information System Engineer, Ph.D. Website: http://matthieubrucher.developpez.com/ Blogs: http://matt.eifelle.com and http://blog.developpez.com/?blog=92 LinkedIn: http://www.linkedin.com/in/matthieubrucher
Hi Matthieu, Thanks for your quick reply! I did not find this page earlier. I have studied the first page and now I realize again that all my efforts so far (last few days) have crashed on the lack of calling import_array(). Do you have a suggestion for calling import_array() as well? Everywhere I see *that* import_array() needs to be called, but not how... I will take a look at the conversion to a single vector. I hope this can be avoided because I cannot simply change the library. regards Raymond Matthieu Brucher wrote:
Hi,
In fact, it's not that complicated. You may know the way how copying a vector, and this is all you need (http://matt.eifelle.com/2008/01/04/transformingacvectorintoanumpyarra...). You will have to copy your data, it is the safest way to ensure that the data is always valid. For the std::vector<>[], I suggest you convert it to a single vector, as the data inside this "array" is not contiguous and it can thus be cumbersome to create a Numpy array from that.
Once the structure is prepared, you have to allocate the dimensions, but this may be available online.
In case you know that the C++ data won't go away before the Python array, you can always wrap the container (http://matt.eifelle.com/2008/11/04/exposinganarrayinterfacewithswigfor...) with SWIG.
Matthieu
2009/7/30 Raymond de Vries
: Hi everyone,
(I sent this message yesterday as well but somehow it didn't come through...)
I would like to ask your advice how I can use NumPy to wrap my existing C++ library with 2dimensional arrays. I am wrapping the library with swig and I have the typemap function declaration. But now I am struggling a lot to find an implementation for getting access to the 2dimensional arrays in my C++ classes. I have wrapped most of the library but the multidimensional arrays are problematic. From what I have read, I can use NumPy for this..
There is so much information about NumPy, Numeric etc that it is not clear anymore which example I can use best. It's clear that NumPy is probably the way to go.
This is a small example of the class that I would like to wrap: struct MyClass { float array1[2][2]; std::vector<float> array2[2]; };
I've also read that I need to call import_array() but where should I put this?
Is there a small example with NumPy capi to wrap/convert my 2dimensional arrays to python?
Thanks a lot already Raymond _______________________________________________ NumPyDiscussion mailing list NumPyDiscussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpydiscussion
Hi, I'm sorry, I guess I did not search properly before.... For the record, I solved my import_array() question: just need to add %init %{ import_array(); %} and the typemap for the std::vector<double> works ok. Thanks for that! Now the rest... Thanks Raymond Raymond de Vries wrote:
Hi Matthieu,
Thanks for your quick reply! I did not find this page earlier. I have studied the first page and now I realize again that all my efforts so far (last few days) have crashed on the lack of calling import_array(). Do you have a suggestion for calling import_array() as well? Everywhere I see *that* import_array() needs to be called, but not how...
I will take a look at the conversion to a single vector. I hope this can be avoided because I cannot simply change the library.
regards Raymond
Matthieu Brucher wrote:
Hi,
In fact, it's not that complicated. You may know the way how copying a vector, and this is all you need (http://matt.eifelle.com/2008/01/04/transformingacvectorintoanumpyarra...). You will have to copy your data, it is the safest way to ensure that the data is always valid. For the std::vector<>[], I suggest you convert it to a single vector, as the data inside this "array" is not contiguous and it can thus be cumbersome to create a Numpy array from that.
Once the structure is prepared, you have to allocate the dimensions, but this may be available online.
In case you know that the C++ data won't go away before the Python array, you can always wrap the container (http://matt.eifelle.com/2008/11/04/exposinganarrayinterfacewithswigfor...) with SWIG.
Matthieu
2009/7/30 Raymond de Vries
: Hi everyone,
(I sent this message yesterday as well but somehow it didn't come through...)
I would like to ask your advice how I can use NumPy to wrap my existing C++ library with 2dimensional arrays. I am wrapping the library with swig and I have the typemap function declaration. But now I am struggling a lot to find an implementation for getting access to the 2dimensional arrays in my C++ classes. I have wrapped most of the library but the multidimensional arrays are problematic. From what I have read, I can use NumPy for this..
There is so much information about NumPy, Numeric etc that it is not clear anymore which example I can use best. It's clear that NumPy is probably the way to go.
This is a small example of the class that I would like to wrap: struct MyClass { float array1[2][2]; std::vector<float> array2[2]; };
I've also read that I need to call import_array() but where should I put this?
Is there a small example with NumPy capi to wrap/convert my 2dimensional arrays to python?
Thanks a lot already Raymond _______________________________________________ NumPyDiscussion mailing list NumPyDiscussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpydiscussion
_______________________________________________ NumPyDiscussion mailing list NumPyDiscussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpydiscussion
2009/7/30 Raymond de Vries
Hi,
I'm sorry, I guess I did not search properly before.... For the record, I solved my import_array() question: just need to add %init %{ import_array(); %}
Indeed, the solution is as simple as this ;) The trouble is to find the information!
and the typemap for the std::vector<double> works ok. Thanks for that!
You're welcome.
Now the rest...
Thanks Raymond
Raymond de Vries wrote:
Hi Matthieu,
Thanks for your quick reply! I did not find this page earlier. I have studied the first page and now I realize again that all my efforts so far (last few days) have crashed on the lack of calling import_array(). Do you have a suggestion for calling import_array() as well? Everywhere I see *that* import_array() needs to be called, but not how...
I will take a look at the conversion to a single vector. I hope this can be avoided because I cannot simply change the library.
regards Raymond
Matthieu Brucher wrote:
Hi,
In fact, it's not that complicated. You may know the way how copying a vector, and this is all you need (http://matt.eifelle.com/2008/01/04/transformingacvectorintoanumpyarra...). You will have to copy your data, it is the safest way to ensure that the data is always valid. For the std::vector<>[], I suggest you convert it to a single vector, as the data inside this "array" is not contiguous and it can thus be cumbersome to create a Numpy array from that.
Once the structure is prepared, you have to allocate the dimensions, but this may be available online.
In case you know that the C++ data won't go away before the Python array, you can always wrap the container (http://matt.eifelle.com/2008/11/04/exposinganarrayinterfacewithswigfor...) with SWIG.
Matthieu
2009/7/30 Raymond de Vries
: Hi everyone,
(I sent this message yesterday as well but somehow it didn't come through...)
I would like to ask your advice how I can use NumPy to wrap my existing C++ library with 2dimensional arrays. I am wrapping the library with swig and I have the typemap function declaration. But now I am struggling a lot to find an implementation for getting access to the 2dimensional arrays in my C++ classes. I have wrapped most of the library but the multidimensional arrays are problematic. From what I have read, I can use NumPy for this..
There is so much information about NumPy, Numeric etc that it is not clear anymore which example I can use best. It's clear that NumPy is probably the way to go.
This is a small example of the class that I would like to wrap: struct MyClass { float array1[2][2]; std::vector<float> array2[2]; };
I've also read that I need to call import_array() but where should I put this?
Is there a small example with NumPy capi to wrap/convert my 2dimensional arrays to python?
Thanks a lot already Raymond _______________________________________________ NumPyDiscussion mailing list NumPyDiscussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpydiscussion
_______________________________________________ NumPyDiscussion mailing list NumPyDiscussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpydiscussion
_______________________________________________ NumPyDiscussion mailing list NumPyDiscussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpydiscussion
 Information System Engineer, Ph.D. Website: http://matthieubrucher.developpez.com/ Blogs: http://matt.eifelle.com and http://blog.developpez.com/?blog=92 LinkedIn: http://www.linkedin.com/in/matthieubrucher
Hi
Indeed, the solution is as simple as this ;) The trouble is to find the information!
Yes, there is so much information everywhere. And it's hard to make the first steps.
For the std::vector<>[], I suggest you convert it to a single vector, as the data inside this "array" is not contiguous and it can thus be cumbersome to create a Numpy array from that.
I am now ready to do this. To be certain, 'contiguous' means that the std::vector<>'s are not the same length, right? Would that mean that I'd better use a tuple of lists or so? (or list of lists or so).
thanks for your time Raymond
Once the structure is prepared, you have to allocate the dimensions, but this may be available online.
In case you know that the C++ data won't go away before the Python array, you can always wrap the container (http://matt.eifelle.com/2008/11/04/exposinganarrayinterfacewithswigfor...) with SWIG.
Matthieu
2009/7/30 Raymond de Vries
: Hi everyone,
(I sent this message yesterday as well but somehow it didn't come through...)
I would like to ask your advice how I can use NumPy to wrap my existing C++ library with 2dimensional arrays. I am wrapping the library with swig and I have the typemap function declaration. But now I am struggling a lot to find an implementation for getting access to the 2dimensional arrays in my C++ classes. I have wrapped most of the library but the multidimensional arrays are problematic. From what I have read, I can use NumPy for this..
There is so much information about NumPy, Numeric etc that it is not clear anymore which example I can use best. It's clear that NumPy is probably the way to go.
This is a small example of the class that I would like to wrap: struct MyClass { float array1[2][2]; std::vector<float> array2[2]; };
I've also read that I need to call import_array() but where should I put this?
Is there a small example with NumPy capi to wrap/convert my 2dimensional arrays to python?
Thanks a lot already Raymond _______________________________________________ NumPyDiscussion mailing list NumPyDiscussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpydiscussion
_______________________________________________ NumPyDiscussion mailing list NumPyDiscussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpydiscussion
_______________________________________________ NumPyDiscussion mailing list NumPyDiscussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpydiscussion
2009/7/30 Raymond de Vries
Hi
Indeed, the solution is as simple as this ;) The trouble is to find the information!
Yes, there is so much information everywhere. And it's hard to make the first steps.
For the std::vector<>[], I suggest you convert it to a single vector, as the data inside this "array" is not contiguous and it can thus be cumbersome to create a Numpy array from that.
I am now ready to do this. To be certain, 'contiguous' means that the std::vector<>'s are not the same length, right? Would that mean that I'd better use a tuple of lists or so? (or list of lists or so).
thanks for your time Raymond
Contiguous means that the whole data is in one big chunk. If it is not, you have to rely on other Numpy functions (I don't know all of them, perhaps you will find one that satisfies your need), and the data may then be copied (not sure though). Matthieu  Information System Engineer, Ph.D. Website: http://matthieubrucher.developpez.com/ Blogs: http://matt.eifelle.com and http://blog.developpez.com/?blog=92 LinkedIn: http://www.linkedin.com/in/matthieubrucher
Hi Matthieu, Thanks for the explanation. After having looked at the documentation, I decided to do my own plain Python capi implementation. Thanks for your time Raymond Matthieu Brucher wrote:
2009/7/30 Raymond de Vries
: Hi
Indeed, the solution is as simple as this ;) The trouble is to find the information!
Yes, there is so much information everywhere. And it's hard to make the first steps.
For the std::vector<>[], I suggest you convert it to a single vector, as the data inside this "array" is not contiguous and it can thus be cumbersome to create a Numpy array from that.
I am now ready to do this. To be certain, 'contiguous' means that the std::vector<>'s are not the same length, right? Would that mean that I'd better use a tuple of lists or so? (or list of lists or so).
thanks for your time Raymond
Contiguous means that the whole data is in one big chunk. If it is not, you have to rely on other Numpy functions (I don't know all of them, perhaps you will find one that satisfies your need), and the data may then be copied (not sure though).
Matthieu
Raymond de Vries wrote:
Thanks for the explanation. After having looked at the documentation, I decided to do my own plain Python capi implementation.
That is unlikely to be the best option these days  it's simply too easy to make a type checking and or reference counting error. If SWIG isn't your cup of tea, take a look at Cython or Ctypes  lower level and more control, but still handle much of the book keeping for you. The Cython team is working on better C++ support, though I don't know where they are at with that. Chris  Christopher Barker, Ph.D. Oceanographer Emergency Response Division NOAA/NOS/OR&R (206) 5266959 voice 7600 Sand Point Way NE (206) 5266329 fax Seattle, WA 98115 (206) 5266317 main reception Chris.Barker@noaa.gov
Hi Chris,
Thanks for the explanation. After having looked at the documentation, I decided to do my own plain Python capi implementation.
That is unlikely to be the best option these days  it's simply too easy to make a type checking and or reference counting error.
If SWIG isn't your cup of tea, take a look at Cython or Ctypes  lower level and more control, but still handle much of the book keeping for you.
The Cython team is working on better C++ support, though I don't know where they are at with that.
Oops, I guess I didn't express myself clearly enough: I have used plain Python capi (in my case a list of lists for my 2dimensional arrays) for my typemaps. Sorry for the unclearness. Actually because NumPy is not my cup of tea... Especially because Matthieu suggested that I should convert my data into a contiguous array. So no matter what I use, either swig, cython, or.. I still have the NumPy issue. Do you, or someone else, see another possibility? regards Raymond
Chris
Raymond de Vries wrote:
Oops, I guess I didn't express myself clearly enough: I have used plain Python capi (in my case a list of lists for my 2dimensional arrays) for my typemaps. Sorry for the unclearness. Actually because NumPy is not my cup of tea...
Well, for almost any purpose, numpy arrays are a better fit for a 2d array of numbers than a list of lists, so I'm not sure what kind of tea you like ;)
Especially because Matthieu suggested that I should convert my data into a contiguous array.
a Python list can't share a pointer with your C++ data type anyway, so you'll have to copy anyway  why not make a contiguous array? If you need a "ragged" array, then it's a different story, but a list of numpy arrays may be a better fit.
So no matter what I use, either swig, cython, or.. I still have the NumPy issue.
True, it's probably best to decide what sort of representation you want in Python, then decide how to build your wrappers. If you have a big 2d array in C++, numpy is the obvious choice. Another choice is a wrapper around your C++ class  don't convert to a python type at all. Depending on what you need to do, it may be OK to lost a lot of functionality numpy give you. This is how the SWIG C++ vector wrappers work, for instance. Chris  Christopher Barker, Ph.D. Oceanographer Emergency Response Division NOAA/NOS/OR&R (206) 5266959 voice 7600 Sand Point Way NE (206) 5266329 fax Seattle, WA 98115 (206) 5266317 main reception Chris.Barker@noaa.gov
participants (3)

Christopher Barker

Matthieu Brucher

Raymond de Vries