![](https://secure.gravatar.com/avatar/b24e93182e89a519546baa7bafe054ed.jpg?s=120&d=mm&r=g)
I have a subclass Bar, a 1-dim array which has some methods and some attributes. One of the attributes is a view of the Bar to permit different shaping. Suppose that 'a' is an instance of 'Bar', which has a method 'show' and a view attribute 'v'. a ^ 15 returns a Bar instance, with its methods but without the attributes. I am attempt to change this, Bar has a method __xor__, see below: def __xor__(self, other): ''' Exclusive or: __xor__(x, y) => x ^ y . ''' z= 1 << this loops to the recursion limit result= ArrayType.__xor__(self, other) n= self.n result.n= n result.rowSize= self.rowSize result.show= self.show result.v= _n.reshape(result.view(), (n*n, n*n)) return result Could anyone suggest a workaround please? Colin W.
![](https://secure.gravatar.com/avatar/af6c39d6943bd4b0e1fde23161e7bb8c.jpg?s=120&d=mm&r=g)
I see the same strange result. Here is a minimal code example to demonstrate: import numpy as N class Bar(N.ndarray): v = 0. def __new__(cls, *args, **kwargs): print "running new" return super(Bar, cls).__new__(cls, *args) def __init__(self, *args, **kwargs): print "running init" self[:] = 0 self.v = 3 In [88]: b = Bar(3) running new running init In [89]: b Out[89]: Bar([0, 0, 0]) In [90]: b.v Out[90]: 3 In [91]: c = b+1 In [92]: c.v Out[92]: 0.0 However, if I do b[:] = 1, everything works fine. Stéfan On Fri, Feb 24, 2006 at 10:56:02AM -0500, Colin J. Williams wrote:
![](https://secure.gravatar.com/avatar/49df8cd4b1b6056c727778925f86147a.jpg?s=120&d=mm&r=g)
Stefan van der Walt wrote:
It's only strange if you have assumptions your not revealing. Here's the deal. Neither the __init__ method nor the __new__ method are called for c = b+1. So, your wondering how the Bar object got created then right? Well, it got created as a subclass of ndarray in PyArray_NewFromDescr. The __init__ and __new__ methods are not called because they may have arbitrary signatures. Instead, the __array_finalize__ method is always called. So, you should use that instead of __init__. The __array_finalize__ method always receives the argument of the "parent" object. Thus in your case. def __array_finalize__(self, parent): self.v = 3 would do what you want. -Travis
![](https://secure.gravatar.com/avatar/af6c39d6943bd4b0e1fde23161e7bb8c.jpg?s=120&d=mm&r=g)
On Fri, Feb 24, 2006 at 06:40:16PM -0700, Travis Oliphant wrote:
That doesn't seem to work. __array_finalize__ isn't called when the object is initially constructed: In [14]: b = Bar(2) running new In [15]: b.v Out[15]: 0.0 In [16]: b=b+1 In [17]: b.v Out[17]: 3 Should a person then call __array_finalize__ from __init__? Stéfan
![](https://secure.gravatar.com/avatar/b24e93182e89a519546baa7bafe054ed.jpg?s=120&d=mm&r=g)
Travis Oliphant wrote:
Why this change in style from the the common Python idom of __new__, __init__, with the same signature to __new__, __array_finalize__ with possibly different signatures? Incidentally, what are the signatures? The doc string is empty: [Dbg]>>> _n.ndarray.__array_finalize__.__doc__ [Dbg]>>> Colin W.
![](https://secure.gravatar.com/avatar/49df8cd4b1b6056c727778925f86147a.jpg?s=120&d=mm&r=g)
I don't see it as a change in style but adding a capability to the ndarray subclass. The problem is that arrays can be created in many ways (slicing, ufuncs, etc). Not all of these ways should go through the __new__/__init__ -- style creation mechanism. Try inheriting from a float builtin and add attributes. Then add your float to an instance of your new class and see what happens. You will get a float-type on the output. This is the essence of Paul's insight that sub-classing is rarely useful because you end up having to re-define all the operators anyway to return the value that you want. He knows whereof he speaks as well, because he wrote MA and UserArray and had experience with Python sub-classing. I wanted a mechanism to make it easier to sub-class arrays and have the operators return your object if possible (including all of it's attributes). Thus, __array_priority__ (a floating point attribute) __array_finalize__ (a method called on internal construction of the array wrapper). were invented (along with __array_wrap__ which any class can define to have their objects survive ufuncs). It was easy enough to see where to call __array_finalize__ in the C-code if somewhat difficult to explain (and get exception handling to work because of my initial over-thinking). The signature is just __array_finalize__(self, parent): return i.e. any return value is ignored (but exceptions are caught). I've used the feature succesfully on at least 3-subclasses (chararray, memmap, and matrix) and so I'm actually pretty happy with it. __new__ and __init__ are still relevant for constructing your brand-new object. The __array_finalize__ function is just what the internal contructor that acutally allocates memory will always call to let you set final attributes *every* time your sub-class gets created. -Travis
![](https://secure.gravatar.com/avatar/e815306c7885ec5d66dafb2f927c5faa.jpg?s=120&d=mm&r=g)
What do these do again? That is, how are they used internally? What happens if they're not used? If I (or anyone else) is to put together a wiki page about this (see the other thread I just emailed to, please), getting good concise descriptions of what ndarray subclasses need to do/can do would be very helpful. Zach
![](https://secure.gravatar.com/avatar/49df8cd4b1b6056c727778925f86147a.jpg?s=120&d=mm&r=g)
Colin J. Williams wrote:
The ndarray handles sub-classing a little-bit differently. All array constructors go through the same C-code call which can create sub-classes as well. (it's what gets called by ndarray.__new__). If there is a parent object, then additionally, the __array_finalize__(self, parent) method is called right-after creation of the sub-class. This is where attributes should be finalized. But, care must be taken in this code so that a recursion is not setup If this mechanism is not sufficient for you, then you need to use a container class (for this reason UserArray has been resurrected to serve as a default container class model---it needs more testing, however). The problem __array_finalize__ helps fix is how to get subclasses to work well without having to re-define every single special method like UserArray does. For the most part it seems to work, but I suppose it creates a few surprises if you are not aware of what is going on. The most important thing to remember is that attributes are not automatically carried over to new instances because new instances can be created without every calling __new__ or __init__. I'm sure this mechanism can be improved upon and I welcome suggestions.
Look at the __array_finalize__ method in defmatrix.py for ideas about how it can be used. -Travis
![](https://secure.gravatar.com/avatar/b24e93182e89a519546baa7bafe054ed.jpg?s=120&d=mm&r=g)
Travis Oliphant wrote:
Thanks for this. Does this mean that whenever we subclass ndarray, we should use __array_finalize__ (with its additional 'parent' parameter) instead of Python's usual __init__? It would help if you could clarify the role of 'parent'. [Dbg]>>> h(self.__array_finalize__) Help on built-in function __array_finalize__: __array_finalize__(...) Is parent the next type up in the type hierarchy? If so, can this not be determined from self.__class__? I've tried a similar operation with the Python library's sets.Set. There, __init__ is called, ensuring that the expression is of the appropriate sub-type.
def __array_finalize__(self, obj): ndim = self.ndim if ndim == 0: self.shape = (1,1) elif ndim == 1: self.shape = (1,self.shape[0]) return These are functions for which one would use __init__ in numarray. This doesn't describe or illustrate the role or purpose of the parent object. Colin W.
![](https://secure.gravatar.com/avatar/af6c39d6943bd4b0e1fde23161e7bb8c.jpg?s=120&d=mm&r=g)
I see the same strange result. Here is a minimal code example to demonstrate: import numpy as N class Bar(N.ndarray): v = 0. def __new__(cls, *args, **kwargs): print "running new" return super(Bar, cls).__new__(cls, *args) def __init__(self, *args, **kwargs): print "running init" self[:] = 0 self.v = 3 In [88]: b = Bar(3) running new running init In [89]: b Out[89]: Bar([0, 0, 0]) In [90]: b.v Out[90]: 3 In [91]: c = b+1 In [92]: c.v Out[92]: 0.0 However, if I do b[:] = 1, everything works fine. Stéfan On Fri, Feb 24, 2006 at 10:56:02AM -0500, Colin J. Williams wrote:
![](https://secure.gravatar.com/avatar/49df8cd4b1b6056c727778925f86147a.jpg?s=120&d=mm&r=g)
Stefan van der Walt wrote:
It's only strange if you have assumptions your not revealing. Here's the deal. Neither the __init__ method nor the __new__ method are called for c = b+1. So, your wondering how the Bar object got created then right? Well, it got created as a subclass of ndarray in PyArray_NewFromDescr. The __init__ and __new__ methods are not called because they may have arbitrary signatures. Instead, the __array_finalize__ method is always called. So, you should use that instead of __init__. The __array_finalize__ method always receives the argument of the "parent" object. Thus in your case. def __array_finalize__(self, parent): self.v = 3 would do what you want. -Travis
![](https://secure.gravatar.com/avatar/af6c39d6943bd4b0e1fde23161e7bb8c.jpg?s=120&d=mm&r=g)
On Fri, Feb 24, 2006 at 06:40:16PM -0700, Travis Oliphant wrote:
That doesn't seem to work. __array_finalize__ isn't called when the object is initially constructed: In [14]: b = Bar(2) running new In [15]: b.v Out[15]: 0.0 In [16]: b=b+1 In [17]: b.v Out[17]: 3 Should a person then call __array_finalize__ from __init__? Stéfan
![](https://secure.gravatar.com/avatar/b24e93182e89a519546baa7bafe054ed.jpg?s=120&d=mm&r=g)
Travis Oliphant wrote:
Why this change in style from the the common Python idom of __new__, __init__, with the same signature to __new__, __array_finalize__ with possibly different signatures? Incidentally, what are the signatures? The doc string is empty: [Dbg]>>> _n.ndarray.__array_finalize__.__doc__ [Dbg]>>> Colin W.
![](https://secure.gravatar.com/avatar/49df8cd4b1b6056c727778925f86147a.jpg?s=120&d=mm&r=g)
I don't see it as a change in style but adding a capability to the ndarray subclass. The problem is that arrays can be created in many ways (slicing, ufuncs, etc). Not all of these ways should go through the __new__/__init__ -- style creation mechanism. Try inheriting from a float builtin and add attributes. Then add your float to an instance of your new class and see what happens. You will get a float-type on the output. This is the essence of Paul's insight that sub-classing is rarely useful because you end up having to re-define all the operators anyway to return the value that you want. He knows whereof he speaks as well, because he wrote MA and UserArray and had experience with Python sub-classing. I wanted a mechanism to make it easier to sub-class arrays and have the operators return your object if possible (including all of it's attributes). Thus, __array_priority__ (a floating point attribute) __array_finalize__ (a method called on internal construction of the array wrapper). were invented (along with __array_wrap__ which any class can define to have their objects survive ufuncs). It was easy enough to see where to call __array_finalize__ in the C-code if somewhat difficult to explain (and get exception handling to work because of my initial over-thinking). The signature is just __array_finalize__(self, parent): return i.e. any return value is ignored (but exceptions are caught). I've used the feature succesfully on at least 3-subclasses (chararray, memmap, and matrix) and so I'm actually pretty happy with it. __new__ and __init__ are still relevant for constructing your brand-new object. The __array_finalize__ function is just what the internal contructor that acutally allocates memory will always call to let you set final attributes *every* time your sub-class gets created. -Travis
![](https://secure.gravatar.com/avatar/e815306c7885ec5d66dafb2f927c5faa.jpg?s=120&d=mm&r=g)
What do these do again? That is, how are they used internally? What happens if they're not used? If I (or anyone else) is to put together a wiki page about this (see the other thread I just emailed to, please), getting good concise descriptions of what ndarray subclasses need to do/can do would be very helpful. Zach
![](https://secure.gravatar.com/avatar/49df8cd4b1b6056c727778925f86147a.jpg?s=120&d=mm&r=g)
Colin J. Williams wrote:
The ndarray handles sub-classing a little-bit differently. All array constructors go through the same C-code call which can create sub-classes as well. (it's what gets called by ndarray.__new__). If there is a parent object, then additionally, the __array_finalize__(self, parent) method is called right-after creation of the sub-class. This is where attributes should be finalized. But, care must be taken in this code so that a recursion is not setup If this mechanism is not sufficient for you, then you need to use a container class (for this reason UserArray has been resurrected to serve as a default container class model---it needs more testing, however). The problem __array_finalize__ helps fix is how to get subclasses to work well without having to re-define every single special method like UserArray does. For the most part it seems to work, but I suppose it creates a few surprises if you are not aware of what is going on. The most important thing to remember is that attributes are not automatically carried over to new instances because new instances can be created without every calling __new__ or __init__. I'm sure this mechanism can be improved upon and I welcome suggestions.
Look at the __array_finalize__ method in defmatrix.py for ideas about how it can be used. -Travis
![](https://secure.gravatar.com/avatar/b24e93182e89a519546baa7bafe054ed.jpg?s=120&d=mm&r=g)
Travis Oliphant wrote:
Thanks for this. Does this mean that whenever we subclass ndarray, we should use __array_finalize__ (with its additional 'parent' parameter) instead of Python's usual __init__? It would help if you could clarify the role of 'parent'. [Dbg]>>> h(self.__array_finalize__) Help on built-in function __array_finalize__: __array_finalize__(...) Is parent the next type up in the type hierarchy? If so, can this not be determined from self.__class__? I've tried a similar operation with the Python library's sets.Set. There, __init__ is called, ensuring that the expression is of the appropriate sub-type.
def __array_finalize__(self, obj): ndim = self.ndim if ndim == 0: self.shape = (1,1) elif ndim == 1: self.shape = (1,self.shape[0]) return These are functions for which one would use __init__ in numarray. This doesn't describe or illustrate the role or purpose of the parent object. Colin W.
participants (5)
-
Colin J. Williams
-
Stefan van der Walt
-
Travis Oliphant
-
Travis Oliphant
-
Zachary Pincus