[Numpy-discussion] Trying to implement the array interface

[Travis E. Oliphant]

Mark Asbach wrote:
> Hi there,
>
> I'm currently extending the Python wrapper for the Open Computer 
> Vision Library (opencv) with the goal to interface numerical libraries 
> as seemless as possible. Unfortunately, it doesn't seem to be that 
> easy ;-)
>
> What I've done so far:
>
> - Added an __array_interface__ property to the Python side of OpenCV 
> data structures (matrices and images) that uses version 3 of the 
> interface definition and supplies the keys 'version', 'shape', 
> 'typestr', 'data' and in some cases 'strides' when we have 
> non-continuos memory layout. I think, I'm compatible to 
> http://numpy.scipy.org/array_interface.shtml .
Great.   This is a good first step.
> - Added parsing the __array_interface__ of Python objects passed to 
> OpenCV methods. I'm a bit unsure of how to use the C/C++ side (array 
> struct) and if I can expect it to be there (for example: I don't 
> provide one with OpenCV). Since I intend to keep OpenCV independent of 
> numpy, calling functions from numpy.h is not an option, as far as I 
> can see.
You can't expect the __array_struct__ property to be there, but if it 
is, it allows you to get all the information you need with one attribute 
lookup (albeit in C) instead of many.
>
> The stuff described above is in the head revision of OpenCV, 
> accessible via "svn co 
> https://opencvlibrary.svn.sourceforge.net/svnroot/opencvlibrary/trunk/opencv". 
>
>
> I've tried using the following packages with OpenCV this way:
> - numpy (1.0.4): everything works as expected. This is the most 
> important library for OpenCV users, so this is a good sign.
> - pylab/matplotlib (0.91.2): seems to use numpy / scipy-core. 
> Everything okay.
> - PIL (1.1.6): the array interface (Python side) doesn't adhere to the 
> definition -> no 'version' key, 'data' is string, not a tuple holding 
> the pointer. What to do with this?
That is probably true.   I've worked a bit with PIL to get things to 
work, but haven't followed the project lately to see where it is at.   
One difficulty is that the PIL memory layout can be quite different from 
a NumPy array, and so that would be why the "data" is a string.   The 
best source for implementing consuming of the interface is to look in 
the NumPy source code and look for where it grabs the 
__array_interface__ and/or __array_struct__ attribute and makes use of 
the data found there.
> - Numeric (24.2): I can create arrays from OpenCV datatypes and print 
> them. Converting to other types however always yields 'Cannot convert 
> scalar to float' or 'a float is required'. Strange :-/ Numeric.array 
> instances also don't carry an __array_interface__. I can however 
> convert by using numpy.arrays as intermediate step.
I believe the __array_struct__ property was used instead.     You can 
implement either __array_struct__ or __array_interface__ or both as an 
exporter.   Thus, a consumer that wants to see everything has to consume 
both.   I'm not sure I understand the error you are getting exactly.  
> - Gnuplot (1.7): uses Numeric, so doesn't work as well
> - pymat: didn't check. Seems to use Numeric, test results cover 
> Numeric 23 and Matlab 6.5 only, so this package might be dead?
I don't remember pymat very well.
> - numarray: didn't check. Is there still any relevance of this package? 
A few tools still use it, but it is deprecated (as is Numeric of 
course).   

I'm glad to see you using the __array_interface__ because it will allow 
you to share memory with numXXX arrays.  I'm sure you are also aware of 
the new buffer protocol in Python 2.6 and Python 3.0.   This is the 
approach to take for the future.  NumPy, of course, will support the 
__array_interface__ and __array_struct__ properties for some time.   I'm 
hoping that a new release of NumPy will also support the new Python 
buffer interface.   But, as of yet, it does not.

Best regards,

-Travis