Multiple inheritance from ndarray
I have a swig extension that defines a class that inherits from both a personal C-coded image struct (actImage), and also from Numeric's UserArray. This works very nicely, but I thought that it was about time to upgrade to numpy. The code looks like: from UserArray import * class Image(UserArray, actImage): def __init__(self, *args): actImage.__init__(self, *args) UserArray.__init__(self, self.getArray(), 'd', copy=False, savespace=False) I can't figure out how to convert this to use ndarray, as ndarray doesn't seem to have an __init__ method, merely a __new__. So what's the approved numpy way to handle multiple inheritance? I've a nasty idea that this is a python question that I should know the answer to, but I'm afraid that I don't... R
Since no one has answered this, I am going to take a whack at it. Experts feel free to shoot me down. Here is a sample showing multiple inheritance with a mix of old style and new style classes. I don't claim there is any logic to the code, but it is just for demo purposes. -------------------------------------- from numpy import * class actImage: def __init__(self, colorOrder='RGBA'): self.colorOrder = colorOrder class Image(actImage, ndarray): def __new__(cls, shape=(1024,768), dtype=float32): return ndarray.__new__(cls, shape=shape, dtype=dtype) x = Image() assert isinstance(x[0,1], float32) assert x.colorOrder == 'RGBA' -------------------------------------- Running "help(ndarray)" has some useful info as well. - Charlie On 2/19/06, Robert Lupton <rhl@astro.princeton.edu> wrote:
I have a swig extension that defines a class that inherits from both a personal C-coded image struct (actImage), and also from Numeric's UserArray. This works very nicely, but I thought that it was about time to upgrade to numpy.
The code looks like:
from UserArray import *
class Image(UserArray, actImage): def __init__(self, *args): actImage.__init__(self, *args) UserArray.__init__(self, self.getArray(), 'd', copy=False, savespace=False)
I can't figure out how to convert this to use ndarray, as ndarray doesn't seem to have an __init__ method, merely a __new__.
So what's the approved numpy way to handle multiple inheritance? I've a nasty idea that this is a python question that I should know the answer to, but I'm afraid that I don't...
R
------------------------------------------------------- This SF.net email is sponsored by: Splunk Inc. Do you grep through log files for problems? Stop! Download the new AJAX search engine that makes searching your log files as easy as surfing the web. DOWNLOAD SPLUNK! http://sel.as-us.falkag.net/sel?cmd=lnk&kid=103432&bid=230486&dat=121642 _______________________________________________ Numpy-discussion mailing list Numpy-discussion@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/numpy-discussion
Robert Lupton wrote:
I have a swig extension that defines a class that inherits from both a personal C-coded image struct (actImage), and also from Numeric's UserArray. This works very nicely, but I thought that it was about time to upgrade to numpy.
The code looks like:
from UserArray import *
class Image(UserArray, actImage): def __init__(self, *args): actImage.__init__(self, *args) UserArray.__init__(self, self.getArray(), 'd', copy=False, savespace=False)
I can't figure out how to convert this to use ndarray, as ndarray doesn't seem to have an __init__ method, merely a __new__.
Yes, the ndarray method doesn't have an __init__ method (so you don't have to call it). What you need to do is write a __new__ method for your class. However, with multiple-inheritance the details matter. You may actually want to have your C-coded actImage class inherit (in C) from the ndarray. If you would like help on that approach let me know (I'll need to understand your actImage a bit better). But, this can all be done in Python, too, but it is a bit of effort to make sure things get created correctly. Perhaps it might make sense to actually include a slightly modified form of the UserArray in NumPy as a standard "container-class" (instead of a sub-class) of the ndarray. In reality, a container class like UserArray and a sub-class are different things. Here's an outline of what you need to do. This is, of course, untested.... For example, I don't really know what actImage is. from numpy import ndarray, array class Image(ndarray, actImage): def __new__(subtype, *args) act1 = actImage.__new__(actImage, *args) actImage.__init__(act1, *args) arr = array(act1.getArray(), 'd', copy=False) self = arr.view(subtype) # you might need to copy attributes from act1 over to self here... return self The problem here, is that apparently you are creating the array first in actImage.__init__ and then passing it to UserArray. The ndarray constructor wants to either create the array itself or use a buffer-exposing object to use as the memory. Keep us posted as your example is a good one that can help us all learn. -Travis
On Wed, Feb 22, 2006 at 08:58:28PM -0700, Travis Oliphant wrote:
Here's an outline of what you need to do. This is, of course, untested.... For example, I don't really know what actImage is.
from numpy import ndarray, array
class Image(ndarray, actImage): def __new__(subtype, *args) act1 = actImage.__new__(actImage, *args) actImage.__init__(act1, *args) arr = array(act1.getArray(), 'd', copy=False) self = arr.view(subtype) # you might need to copy attributes from act1 over to self here... return self
This is probably the right place to use super, i.e.: def __new__(subtype, *args): act1 = super(Image, subtype).__new__(subtype, *args) ... def __init__(self, *args): super(Image, self).__init__(*args) The attached script shows how multiple inheritance runs through different classes. Stéfan
I've finally found time to return to this problem. Travis's made the suggestion that I could use code along the lines of:
class Image(ndarray, actImage): def __new__(subtype, *args) act1 = actImage.__new__(actImage, *args) actImage.__init__(act1, *args) arr = array(act1.getArray(), 'd', copy=False) self = arr.view(subtype)
return self
and this makes sense. Unfortunately, the arr.view(subtype) fails: TypeError: Cannot change descriptor for objectarray. [incidently, is that a typo for "object array"; the string's built by "C string" "concatenation"). Ideas? The C type "actImage" looks like: typedef struct actImage { int nrow; // number of rows int ncol; // number of columns void *base; // allocated memory, if owned by this object int baseRefCount; // reference counter for base double **rows; // Fluxes for pixels PyArrayObject *_array; // pointer to Numeric array; used by actNumPy } actImage; The _array is set in the C actImage constructor with: im->_array = (PyArrayObject *)PyArray_FromDimsAndData(2, dims, PyArray_DOUBLE, (void *)im->base); and getArray is a wrapper around: PyObject *actImageGetNumPy(const actImage *im) { Py_XINCREF(im->_array); return PyArray_Return(im->_array); } R
Robert Lupton wrote:
I have a swig extension that defines a class that inherits from both a personal C-coded image struct (actImage), and also from Numeric's UserArray. This works very nicely, but I thought that it was about time to upgrade to numpy.
The code looks like:
from UserArray import *
class Image(UserArray, actImage): def __init__(self, *args): actImage.__init__(self, *args) UserArray.__init__(self, self.getArray(), 'd', copy=False, savespace=False)
I can't figure out how to convert this to use ndarray, as ndarray doesn't seem to have an __init__ method, merely a __new__.
Yes, the ndarray method doesn't have an __init__ method (so you don't have to call it).
What you need to do is write a __new__ method for your class. However, with multiple-inheritance the details matter. You may actually want to have your C-coded actImage class inherit (in C) from the ndarray. If you would like help on that approach let me know (I'll need to understand your actImage a bit better). But, this can all be done in Python, too, but it is a bit of effort to make sure things get created correctly. Perhaps it might make sense to actually include a slightly modified form of the UserArray in NumPy as a standard "container-class" (instead of a sub-class) of the ndarray. In reality, a container class like UserArray and a sub-class are different things.
Here's an outline of what you need to do. This is, of course, untested.... For example, I don't really know what actImage is.
from numpy import ndarray, array
class Image(ndarray, actImage): def __new__(subtype, *args) act1 = actImage.__new__(actImage, *args) actImage.__init__(act1, *args) arr = array(act1.getArray(), 'd', copy=False) self = arr.view(subtype) # you might need to copy attributes from act1 over to self here... return self
The problem here, is that apparently you are creating the array first in actImage.__init__ and then passing it to UserArray. The ndarray constructor wants to either create the array itself or use a buffer-exposing object to use as the memory. Keep us posted as your example is a good one that can help us all learn.
-Travis
Robert Lupton wrote:
I've finally found time to return to this problem. Travis's made the suggestion that I could use code along the lines of:
class Image(ndarray, actImage): def __new__(subtype, *args) act1 = actImage.__new__(actImage, *args) actImage.__init__(act1, *args) arr = array(act1.getArray(), 'd', copy=False) self = arr.view(subtype)
return self
and this makes sense. Unfortunately, the arr.view(subtype) fails: TypeError: Cannot change descriptor for objectarray. [incidently, is that a typo for "object array"; the string's built by "C string" "concatenation"). Yes, I think so. Hmm, it looks like you have object arrays floating around which seems strange.
But, as I thought more about multiple inheritance I realized that there really is a case for a container class like UserArray that you can inherit from that only "wraps" the ndarray and does not try to inherit from it. The analogy I see with C-code is that a container class has a pointer to an array object structure, while a sub-class has the arrayobject structure itself as part of the new subclass. So, I added back the UserArray to numpy. Now, you can inherit from numpy.lib.UserArray.UserArray just like before.
Ideas?
The C type "actImage" looks like: typedef struct actImage { int nrow; // number of rows int ncol; // number of columns void *base; // allocated memory, if owned by this object int baseRefCount; // reference counter for base double **rows; // Fluxes for pixels PyArrayObject *_array; // pointer to Numeric array; used by actNumPy } actImage; It looks like you have a "container" class by default here. You could, if you wanted make this a subclass by definining actImage as
typedef struct actImage { PyArrayObject _array; int nrow; // redundant int ncol; // redundant void * base; // also seems redundant int baseRefCount; // perhaps not necessary either double **rows; // truly new information } actImage; Now, a pointer to actImage could be cast to a (PyArrayObject *) and used wherever arrayobjects are expected and work just fine. I'm not sure why you have a different pointer for allocated memory which could It seems to me that you should create a sub-type in C of the ndarray rather than a container class. But, a container class would also continue to work. With the re-addition of UserArray it would work in much the same way as well.
The _array is set in the C actImage constructor with: im->_array = (PyArrayObject *)PyArray_FromDimsAndData(2, dims, PyArray_DOUBLE, (void *)im->base);
For the subclass you would need to do a little more work in the constructor because PyArray_FromDims returns only a filled-in PyArrayObject structure. You could copy all the elements of this array over (this wouldn't copy data just pointers), and then delete the PyArrayObject structure (don't use DECREF, just tp_free). Then you could fill in the rest of the actImage structure.
and getArray is a wrapper around:
PyObject *actImageGetNumPy(const actImage *im) { Py_XINCREF(im->_array); return PyArray_Return(im->_array); }
You could use the new tp_members structure here which allows you to get access to parts of your C-structure very easily. The easiest transition path is to just use the re-added UserArray as before. If you want to learn to sub-class, that could be done as well. But, there are no sub-classes in C that I know of yet, so although it is possible, there may be some subtle issues uncovered in the process. -Travis
Thanks Travis.
Unfortunately, the arr.view(subtype) fails: TypeError: Cannot change descriptor for objectarray. [incidently, is that a typo for "object array"; the string's built by "C string" "concatenation")?]
Yes, I think so. Hmm, it looks like you have object arrays floating around which seems strange.
Do you know why I've acquired object arrays? I don't understand enough numpy guts to figure it out myself. If it's an interesting question, I can give you a complete tarball.
So, I added back the UserArray to numpy. Now, you can inherit from numpy.lib.UserArray.UserArray just like before.
This is in the latest svn version? R
The latest version of swig (1.3.28 or 1.3.29) has broken my multiple-inheritance-from-C-and-numpy application; more specifically, it generates an infinite loop in numpy-land. I'm using numpy (0.9.6), and here's the offending code. Ideas anyone? I've pasted the crucial part of numpy.lib.UserArray onto the end of this message (how do I know? because you can replace the "from numpy.lib.UserArray" with this, and the problem persists). ##################################################### from numpy.lib.UserArray import * import types class myImage(types.ObjectType): def __init__(self, *args): this = None try: self.this.append(this) except: self.this = this class Image(UserArray, myImage): def __init__(self, *args): myImage.__init__(self, *args) ##################################################### The symptoms are: from recursionBug import *; Image(myImage()) ------------------------------------------------------------ Traceback (most recent call last): File "<stdin>", line 1, in ? File "recursionBug.py", line 32, in __init__ myImage.__init__(self, *args) File "recursionBug.py", line 26, in __init__ except: self.this = this File "/sw/lib/python2.4/site-packages/numpy/lib/UserArray.py", line 187, in __setattr__ self.array.__setattr__(attr, value) File "/sw/lib/python2.4/site-packages/numpy/lib/UserArray.py", line 193, in __getattr__ return self.array.__getattribute__(attr) ... File "/sw/lib/python2.4/site-packages/numpy/lib/UserArray.py", line 193, in __getattr__ return self.array.__getattribute__(attr) File "/sw/lib/python2.4/site-packages/numpy/lib/UserArray.py", line 193, in __getattr__ return self.array.__getattribute__(attr) RuntimeError: maximum recursion depth exceeded The following stripped down piece of numpy seems to be the problem: class UserArray(object): def __setattr__(self,attr,value): try: self.array.__setattr__(attr, value) except AttributeError: object.__setattr__(self, attr, value) # Only called after other approaches fail. def __getattr__(self,attr): return self.array.__getattribute__(attr) R
The latest version of swig (1.3.28 or 1.3.29) has broken my multiple-inheritance-from-C-and-numpy application; more specifically, it generates an infinite loop in numpy-land. I'm using numpy (0.9.6), and here's the offending code. Ideas anyone? I've pasted the crucial part of numpy.lib.UserArray onto the end of this message (how do I know? because you can replace the "from numpy.lib.UserArray" with this, and the problem persists). This is a problem in the getattr code of UserArray. This is fixed in SVN. But, you can just replace the getattr code in UserArray.py with
Robert Lupton wrote: the following: def __getattr__(self,attr): if (attr == 'array'): return object.__getattr__(self, attr) return self.array.__getattribute__(attr) Thanks for finding and reporting this. -Travis
participants (4)
-
Charlie Moad -
Robert Lupton -
Stefan van der Walt -
Travis Oliphant