Multiple inheritance from ndarray
![](https://secure.gravatar.com/avatar/433961f440fcf7d4b88b80239b9e3139.jpg?s=120&d=mm&r=g)
I have a swig extension that defines a class that inherits from both a personal C-coded image struct (actImage), and also from Numeric's UserArray. This works very nicely, but I thought that it was about time to upgrade to numpy. The code looks like: from UserArray import * class Image(UserArray, actImage): def __init__(self, *args): actImage.__init__(self, *args) UserArray.__init__(self, self.getArray(), 'd', copy=False, savespace=False) I can't figure out how to convert this to use ndarray, as ndarray doesn't seem to have an __init__ method, merely a __new__. So what's the approved numpy way to handle multiple inheritance? I've a nasty idea that this is a python question that I should know the answer to, but I'm afraid that I don't... R
![](https://secure.gravatar.com/avatar/2748c5c3129451e26aa262b20d62d39a.jpg?s=120&d=mm&r=g)
Since no one has answered this, I am going to take a whack at it. Experts feel free to shoot me down. Here is a sample showing multiple inheritance with a mix of old style and new style classes. I don't claim there is any logic to the code, but it is just for demo purposes. -------------------------------------- from numpy import * class actImage: def __init__(self, colorOrder='RGBA'): self.colorOrder = colorOrder class Image(actImage, ndarray): def __new__(cls, shape=(1024,768), dtype=float32): return ndarray.__new__(cls, shape=shape, dtype=dtype) x = Image() assert isinstance(x[0,1], float32) assert x.colorOrder == 'RGBA' -------------------------------------- Running "help(ndarray)" has some useful info as well. - Charlie On 2/19/06, Robert Lupton <rhl@astro.princeton.edu> wrote:
![](https://secure.gravatar.com/avatar/49df8cd4b1b6056c727778925f86147a.jpg?s=120&d=mm&r=g)
Robert Lupton wrote:
Yes, the ndarray method doesn't have an __init__ method (so you don't have to call it). What you need to do is write a __new__ method for your class. However, with multiple-inheritance the details matter. You may actually want to have your C-coded actImage class inherit (in C) from the ndarray. If you would like help on that approach let me know (I'll need to understand your actImage a bit better). But, this can all be done in Python, too, but it is a bit of effort to make sure things get created correctly. Perhaps it might make sense to actually include a slightly modified form of the UserArray in NumPy as a standard "container-class" (instead of a sub-class) of the ndarray. In reality, a container class like UserArray and a sub-class are different things. Here's an outline of what you need to do. This is, of course, untested.... For example, I don't really know what actImage is. from numpy import ndarray, array class Image(ndarray, actImage): def __new__(subtype, *args) act1 = actImage.__new__(actImage, *args) actImage.__init__(act1, *args) arr = array(act1.getArray(), 'd', copy=False) self = arr.view(subtype) # you might need to copy attributes from act1 over to self here... return self The problem here, is that apparently you are creating the array first in actImage.__init__ and then passing it to UserArray. The ndarray constructor wants to either create the array itself or use a buffer-exposing object to use as the memory. Keep us posted as your example is a good one that can help us all learn. -Travis
![](https://secure.gravatar.com/avatar/af6c39d6943bd4b0e1fde23161e7bb8c.jpg?s=120&d=mm&r=g)
On Wed, Feb 22, 2006 at 08:58:28PM -0700, Travis Oliphant wrote:
This is probably the right place to use super, i.e.: def __new__(subtype, *args): act1 = super(Image, subtype).__new__(subtype, *args) ... def __init__(self, *args): super(Image, self).__init__(*args) The attached script shows how multiple inheritance runs through different classes. Stéfan
![](https://secure.gravatar.com/avatar/433961f440fcf7d4b88b80239b9e3139.jpg?s=120&d=mm&r=g)
I've finally found time to return to this problem. Travis's made the suggestion that I could use code along the lines of:
and this makes sense. Unfortunately, the arr.view(subtype) fails: TypeError: Cannot change descriptor for objectarray. [incidently, is that a typo for "object array"; the string's built by "C string" "concatenation"). Ideas? The C type "actImage" looks like: typedef struct actImage { int nrow; // number of rows int ncol; // number of columns void *base; // allocated memory, if owned by this object int baseRefCount; // reference counter for base double **rows; // Fluxes for pixels PyArrayObject *_array; // pointer to Numeric array; used by actNumPy } actImage; The _array is set in the C actImage constructor with: im->_array = (PyArrayObject *)PyArray_FromDimsAndData(2, dims, PyArray_DOUBLE, (void *)im->base); and getArray is a wrapper around: PyObject *actImageGetNumPy(const actImage *im) { Py_XINCREF(im->_array); return PyArray_Return(im->_array); } R
![](https://secure.gravatar.com/avatar/49df8cd4b1b6056c727778925f86147a.jpg?s=120&d=mm&r=g)
Robert Lupton wrote:
But, as I thought more about multiple inheritance I realized that there really is a case for a container class like UserArray that you can inherit from that only "wraps" the ndarray and does not try to inherit from it. The analogy I see with C-code is that a container class has a pointer to an array object structure, while a sub-class has the arrayobject structure itself as part of the new subclass. So, I added back the UserArray to numpy. Now, you can inherit from numpy.lib.UserArray.UserArray just like before.
typedef struct actImage { PyArrayObject _array; int nrow; // redundant int ncol; // redundant void * base; // also seems redundant int baseRefCount; // perhaps not necessary either double **rows; // truly new information } actImage; Now, a pointer to actImage could be cast to a (PyArrayObject *) and used wherever arrayobjects are expected and work just fine. I'm not sure why you have a different pointer for allocated memory which could It seems to me that you should create a sub-type in C of the ndarray rather than a container class. But, a container class would also continue to work. With the re-addition of UserArray it would work in much the same way as well.
For the subclass you would need to do a little more work in the constructor because PyArray_FromDims returns only a filled-in PyArrayObject structure. You could copy all the elements of this array over (this wouldn't copy data just pointers), and then delete the PyArrayObject structure (don't use DECREF, just tp_free). Then you could fill in the rest of the actImage structure.
You could use the new tp_members structure here which allows you to get access to parts of your C-structure very easily. The easiest transition path is to just use the re-added UserArray as before. If you want to learn to sub-class, that could be done as well. But, there are no sub-classes in C that I know of yet, so although it is possible, there may be some subtle issues uncovered in the process. -Travis
![](https://secure.gravatar.com/avatar/433961f440fcf7d4b88b80239b9e3139.jpg?s=120&d=mm&r=g)
Thanks Travis.
Do you know why I've acquired object arrays? I don't understand enough numpy guts to figure it out myself. If it's an interesting question, I can give you a complete tarball.
So, I added back the UserArray to numpy. Now, you can inherit from numpy.lib.UserArray.UserArray just like before.
This is in the latest svn version? R
![](https://secure.gravatar.com/avatar/433961f440fcf7d4b88b80239b9e3139.jpg?s=120&d=mm&r=g)
The latest version of swig (1.3.28 or 1.3.29) has broken my multiple-inheritance-from-C-and-numpy application; more specifically, it generates an infinite loop in numpy-land. I'm using numpy (0.9.6), and here's the offending code. Ideas anyone? I've pasted the crucial part of numpy.lib.UserArray onto the end of this message (how do I know? because you can replace the "from numpy.lib.UserArray" with this, and the problem persists). ##################################################### from numpy.lib.UserArray import * import types class myImage(types.ObjectType): def __init__(self, *args): this = None try: self.this.append(this) except: self.this = this class Image(UserArray, myImage): def __init__(self, *args): myImage.__init__(self, *args) ##################################################### The symptoms are: from recursionBug import *; Image(myImage()) ------------------------------------------------------------ Traceback (most recent call last): File "<stdin>", line 1, in ? File "recursionBug.py", line 32, in __init__ myImage.__init__(self, *args) File "recursionBug.py", line 26, in __init__ except: self.this = this File "/sw/lib/python2.4/site-packages/numpy/lib/UserArray.py", line 187, in __setattr__ self.array.__setattr__(attr, value) File "/sw/lib/python2.4/site-packages/numpy/lib/UserArray.py", line 193, in __getattr__ return self.array.__getattribute__(attr) ... File "/sw/lib/python2.4/site-packages/numpy/lib/UserArray.py", line 193, in __getattr__ return self.array.__getattribute__(attr) File "/sw/lib/python2.4/site-packages/numpy/lib/UserArray.py", line 193, in __getattr__ return self.array.__getattribute__(attr) RuntimeError: maximum recursion depth exceeded The following stripped down piece of numpy seems to be the problem: class UserArray(object): def __setattr__(self,attr,value): try: self.array.__setattr__(attr, value) except AttributeError: object.__setattr__(self, attr, value) # Only called after other approaches fail. def __getattr__(self,attr): return self.array.__getattribute__(attr) R
![](https://secure.gravatar.com/avatar/49df8cd4b1b6056c727778925f86147a.jpg?s=120&d=mm&r=g)
Robert Lupton wrote: the following: def __getattr__(self,attr): if (attr == 'array'): return object.__getattr__(self, attr) return self.array.__getattribute__(attr) Thanks for finding and reporting this. -Travis
![](https://secure.gravatar.com/avatar/2748c5c3129451e26aa262b20d62d39a.jpg?s=120&d=mm&r=g)
Since no one has answered this, I am going to take a whack at it. Experts feel free to shoot me down. Here is a sample showing multiple inheritance with a mix of old style and new style classes. I don't claim there is any logic to the code, but it is just for demo purposes. -------------------------------------- from numpy import * class actImage: def __init__(self, colorOrder='RGBA'): self.colorOrder = colorOrder class Image(actImage, ndarray): def __new__(cls, shape=(1024,768), dtype=float32): return ndarray.__new__(cls, shape=shape, dtype=dtype) x = Image() assert isinstance(x[0,1], float32) assert x.colorOrder == 'RGBA' -------------------------------------- Running "help(ndarray)" has some useful info as well. - Charlie On 2/19/06, Robert Lupton <rhl@astro.princeton.edu> wrote:
![](https://secure.gravatar.com/avatar/49df8cd4b1b6056c727778925f86147a.jpg?s=120&d=mm&r=g)
Robert Lupton wrote:
Yes, the ndarray method doesn't have an __init__ method (so you don't have to call it). What you need to do is write a __new__ method for your class. However, with multiple-inheritance the details matter. You may actually want to have your C-coded actImage class inherit (in C) from the ndarray. If you would like help on that approach let me know (I'll need to understand your actImage a bit better). But, this can all be done in Python, too, but it is a bit of effort to make sure things get created correctly. Perhaps it might make sense to actually include a slightly modified form of the UserArray in NumPy as a standard "container-class" (instead of a sub-class) of the ndarray. In reality, a container class like UserArray and a sub-class are different things. Here's an outline of what you need to do. This is, of course, untested.... For example, I don't really know what actImage is. from numpy import ndarray, array class Image(ndarray, actImage): def __new__(subtype, *args) act1 = actImage.__new__(actImage, *args) actImage.__init__(act1, *args) arr = array(act1.getArray(), 'd', copy=False) self = arr.view(subtype) # you might need to copy attributes from act1 over to self here... return self The problem here, is that apparently you are creating the array first in actImage.__init__ and then passing it to UserArray. The ndarray constructor wants to either create the array itself or use a buffer-exposing object to use as the memory. Keep us posted as your example is a good one that can help us all learn. -Travis
![](https://secure.gravatar.com/avatar/af6c39d6943bd4b0e1fde23161e7bb8c.jpg?s=120&d=mm&r=g)
On Wed, Feb 22, 2006 at 08:58:28PM -0700, Travis Oliphant wrote:
This is probably the right place to use super, i.e.: def __new__(subtype, *args): act1 = super(Image, subtype).__new__(subtype, *args) ... def __init__(self, *args): super(Image, self).__init__(*args) The attached script shows how multiple inheritance runs through different classes. Stéfan
![](https://secure.gravatar.com/avatar/433961f440fcf7d4b88b80239b9e3139.jpg?s=120&d=mm&r=g)
I've finally found time to return to this problem. Travis's made the suggestion that I could use code along the lines of:
and this makes sense. Unfortunately, the arr.view(subtype) fails: TypeError: Cannot change descriptor for objectarray. [incidently, is that a typo for "object array"; the string's built by "C string" "concatenation"). Ideas? The C type "actImage" looks like: typedef struct actImage { int nrow; // number of rows int ncol; // number of columns void *base; // allocated memory, if owned by this object int baseRefCount; // reference counter for base double **rows; // Fluxes for pixels PyArrayObject *_array; // pointer to Numeric array; used by actNumPy } actImage; The _array is set in the C actImage constructor with: im->_array = (PyArrayObject *)PyArray_FromDimsAndData(2, dims, PyArray_DOUBLE, (void *)im->base); and getArray is a wrapper around: PyObject *actImageGetNumPy(const actImage *im) { Py_XINCREF(im->_array); return PyArray_Return(im->_array); } R
![](https://secure.gravatar.com/avatar/49df8cd4b1b6056c727778925f86147a.jpg?s=120&d=mm&r=g)
Robert Lupton wrote:
But, as I thought more about multiple inheritance I realized that there really is a case for a container class like UserArray that you can inherit from that only "wraps" the ndarray and does not try to inherit from it. The analogy I see with C-code is that a container class has a pointer to an array object structure, while a sub-class has the arrayobject structure itself as part of the new subclass. So, I added back the UserArray to numpy. Now, you can inherit from numpy.lib.UserArray.UserArray just like before.
typedef struct actImage { PyArrayObject _array; int nrow; // redundant int ncol; // redundant void * base; // also seems redundant int baseRefCount; // perhaps not necessary either double **rows; // truly new information } actImage; Now, a pointer to actImage could be cast to a (PyArrayObject *) and used wherever arrayobjects are expected and work just fine. I'm not sure why you have a different pointer for allocated memory which could It seems to me that you should create a sub-type in C of the ndarray rather than a container class. But, a container class would also continue to work. With the re-addition of UserArray it would work in much the same way as well.
For the subclass you would need to do a little more work in the constructor because PyArray_FromDims returns only a filled-in PyArrayObject structure. You could copy all the elements of this array over (this wouldn't copy data just pointers), and then delete the PyArrayObject structure (don't use DECREF, just tp_free). Then you could fill in the rest of the actImage structure.
You could use the new tp_members structure here which allows you to get access to parts of your C-structure very easily. The easiest transition path is to just use the re-added UserArray as before. If you want to learn to sub-class, that could be done as well. But, there are no sub-classes in C that I know of yet, so although it is possible, there may be some subtle issues uncovered in the process. -Travis
![](https://secure.gravatar.com/avatar/433961f440fcf7d4b88b80239b9e3139.jpg?s=120&d=mm&r=g)
Thanks Travis.
Do you know why I've acquired object arrays? I don't understand enough numpy guts to figure it out myself. If it's an interesting question, I can give you a complete tarball.
So, I added back the UserArray to numpy. Now, you can inherit from numpy.lib.UserArray.UserArray just like before.
This is in the latest svn version? R
![](https://secure.gravatar.com/avatar/433961f440fcf7d4b88b80239b9e3139.jpg?s=120&d=mm&r=g)
The latest version of swig (1.3.28 or 1.3.29) has broken my multiple-inheritance-from-C-and-numpy application; more specifically, it generates an infinite loop in numpy-land. I'm using numpy (0.9.6), and here's the offending code. Ideas anyone? I've pasted the crucial part of numpy.lib.UserArray onto the end of this message (how do I know? because you can replace the "from numpy.lib.UserArray" with this, and the problem persists). ##################################################### from numpy.lib.UserArray import * import types class myImage(types.ObjectType): def __init__(self, *args): this = None try: self.this.append(this) except: self.this = this class Image(UserArray, myImage): def __init__(self, *args): myImage.__init__(self, *args) ##################################################### The symptoms are: from recursionBug import *; Image(myImage()) ------------------------------------------------------------ Traceback (most recent call last): File "<stdin>", line 1, in ? File "recursionBug.py", line 32, in __init__ myImage.__init__(self, *args) File "recursionBug.py", line 26, in __init__ except: self.this = this File "/sw/lib/python2.4/site-packages/numpy/lib/UserArray.py", line 187, in __setattr__ self.array.__setattr__(attr, value) File "/sw/lib/python2.4/site-packages/numpy/lib/UserArray.py", line 193, in __getattr__ return self.array.__getattribute__(attr) ... File "/sw/lib/python2.4/site-packages/numpy/lib/UserArray.py", line 193, in __getattr__ return self.array.__getattribute__(attr) File "/sw/lib/python2.4/site-packages/numpy/lib/UserArray.py", line 193, in __getattr__ return self.array.__getattribute__(attr) RuntimeError: maximum recursion depth exceeded The following stripped down piece of numpy seems to be the problem: class UserArray(object): def __setattr__(self,attr,value): try: self.array.__setattr__(attr, value) except AttributeError: object.__setattr__(self, attr, value) # Only called after other approaches fail. def __getattr__(self,attr): return self.array.__getattribute__(attr) R
![](https://secure.gravatar.com/avatar/49df8cd4b1b6056c727778925f86147a.jpg?s=120&d=mm&r=g)
Robert Lupton wrote: the following: def __getattr__(self,attr): if (attr == 'array'): return object.__getattr__(self, attr) return self.array.__getattribute__(attr) Thanks for finding and reporting this. -Travis
participants (4)
-
Charlie Moad
-
Robert Lupton
-
Stefan van der Walt
-
Travis Oliphant