I have a subclass Bar, a 1-dim array which has some methods and some attributes. One of the attributes is a view of the Bar to permit different shaping. Suppose that 'a' is an instance of 'Bar', which has a method 'show' and a view attribute 'v'. a ^ 15 returns a Bar instance, with its methods but without the attributes. I am attempt to change this, Bar has a method __xor__, see below: def __xor__(self, other): ''' Exclusive or: __xor__(x, y) => x ^ y . ''' z= 1 << this loops to the recursion limit result= ArrayType.__xor__(self, other) n= self.n result.n= n result.rowSize= self.rowSize result.show= self.show result.v= _n.reshape(result.view(), (n*n, n*n)) return result Could anyone suggest a workaround please? Colin W.
I see the same strange result. Here is a minimal code example to demonstrate: import numpy as N class Bar(N.ndarray): v = 0. def __new__(cls, *args, **kwargs): print "running new" return super(Bar, cls).__new__(cls, *args) def __init__(self, *args, **kwargs): print "running init" self[:] = 0 self.v = 3 In [88]: b = Bar(3) running new running init In [89]: b Out[89]: Bar([0, 0, 0]) In [90]: b.v Out[90]: 3 In [91]: c = b+1 In [92]: c.v Out[92]: 0.0 However, if I do b[:] = 1, everything works fine. Stéfan On Fri, Feb 24, 2006 at 10:56:02AM -0500, Colin J. Williams wrote:
I have a subclass Bar, a 1-dim array which has some methods and some attributes. One of the attributes is a view of the Bar to permit different shaping.
Suppose that 'a' is an instance of 'Bar', which has a method 'show' and a view attribute 'v'.
a ^ 15 returns a Bar instance, with its methods but without the attributes.
I am attempt to change this, Bar has a method __xor__, see below:
def __xor__(self, other): ''' Exclusive or: __xor__(x, y) => x ^ y . ''' z= 1 << this loops to the recursion limit result= ArrayType.__xor__(self, other) n= self.n result.n= n result.rowSize= self.rowSize result.show= self.show result.v= _n.reshape(result.view(), (n*n, n*n)) return result
Could anyone suggest a workaround please?
Colin W.
Stefan van der Walt wrote:
I see the same strange result. Here is a minimal code example to demonstrate:
import numpy as N
class Bar(N.ndarray): v = 0.
def __new__(cls, *args, **kwargs): print "running new" return super(Bar, cls).__new__(cls, *args)
def __init__(self, *args, **kwargs): print "running init" self[:] = 0 self.v = 3
It's only strange if you have assumptions your not revealing. Here's the deal. Neither the __init__ method nor the __new__ method are called for c = b+1. So, your wondering how the Bar object got created then right? Well, it got created as a subclass of ndarray in PyArray_NewFromDescr. The __init__ and __new__ methods are not called because they may have arbitrary signatures. Instead, the __array_finalize__ method is always called. So, you should use that instead of __init__. The __array_finalize__ method always receives the argument of the "parent" object. Thus in your case. def __array_finalize__(self, parent): self.v = 3 would do what you want. -Travis
In [88]: b = Bar(3) running new running init
In [89]: b Out[89]: Bar([0, 0, 0])
In [90]: b.v Out[90]: 3
In [91]: c = b+1
In [92]: c.v Out[92]: 0.0
However, if I do b[:] = 1, everything works fine.
Stéfan
On Fri, Feb 24, 2006 at 10:56:02AM -0500, Colin J. Williams wrote:
I have a subclass Bar, a 1-dim array which has some methods and some attributes. One of the attributes is a view of the Bar to permit different shaping.
Suppose that 'a' is an instance of 'Bar', which has a method 'show' and a view attribute 'v'.
a ^ 15 returns a Bar instance, with its methods but without the attributes.
I am attempt to change this, Bar has a method __xor__, see below:
def __xor__(self, other): ''' Exclusive or: __xor__(x, y) => x ^ y . ''' z= 1 << this loops to the recursion limit result= ArrayType.__xor__(self, other) n= self.n result.n= n result.rowSize= self.rowSize result.show= self.show result.v= _n.reshape(result.view(), (n*n, n*n)) return result
Could anyone suggest a workaround please?
Colin W.
------------------------------------------------------- This SF.Net email is sponsored by xPML, a groundbreaking scripting language that extends applications into web and mobile media. Attend the live webcast and join the prime developer group breaking into this new coding territory! http://sel.as-us.falkag.net/sel?cmd=k&kid0944&bid$1720&dat1642 _______________________________________________ Numpy-discussion mailing list Numpy-discussion@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/numpy-discussion
On Fri, Feb 24, 2006 at 06:40:16PM -0700, Travis Oliphant wrote:
Stefan van der Walt wrote:
I see the same strange result. Here is a minimal code example to demonstrate:
import numpy as N
class Bar(N.ndarray): v = 0.
def __new__(cls, *args, **kwargs): print "running new" return super(Bar, cls).__new__(cls, *args)
def __init__(self, *args, **kwargs): print "running init" self[:] = 0 self.v = 3
It's only strange if you have assumptions your not revealing. Here's the deal.
Neither the __init__ method nor the __new__ method are called for c = b+1.
So, your wondering how the Bar object got created then right? Well, it got created as a subclass of ndarray in PyArray_NewFromDescr.
The __init__ and __new__ methods are not called because they may have arbitrary signatures. Instead, the __array_finalize__ method is always called. So, you should use that instead of __init__.
The __array_finalize__ method always receives the argument of the "parent" object.
Thus in your case.
def __array_finalize__(self, parent): self.v = 3
would do what you want.
That doesn't seem to work. __array_finalize__ isn't called when the object is initially constructed: In [14]: b = Bar(2) running new In [15]: b.v Out[15]: 0.0 In [16]: b=b+1 In [17]: b.v Out[17]: 3 Should a person then call __array_finalize__ from __init__? Stéfan
The __init__ and __new__ methods are not called because they may have arbitrary signatures. Instead, the __array_finalize__ method is always called. So, you should use that instead of __init__.
This is now true in SVN. Previously, __array_finalize__ was not called if the "parent" was NULL. However, now, it is still called with None as
Stefan van der Walt wrote: the value of the first argument. Thus __array_finalize__ will be called whenever ndarray.__new__(<some subclass>,...) is called. -Travis
Travis Oliphant wrote:
Stefan van der Walt wrote:
The __init__ and __new__ methods are not called because they may have arbitrary signatures. Instead, the __array_finalize__ method is always called. So, you should use that instead of __init__.
This is now true in SVN. Previously, __array_finalize__ was not called if the "parent" was NULL. However, now, it is still called with None as the value of the first argument.
Thus __array_finalize__ will be called whenever ndarray.__new__(<some subclass>,...) is called.
Why this change in style from the the common Python idom of __new__, __init__, with the same signature to __new__, __array_finalize__ with possibly different signatures? Incidentally, what are the signatures? The doc string is empty: [Dbg]>>> _n.ndarray.__array_finalize__.__doc__ [Dbg]>>> Colin W.
Travis Oliphant wrote:
Stefan van der Walt wrote:
The __init__ and __new__ methods are not called because they may have arbitrary signatures. Instead, the __array_finalize__ method is always called. So, you should use that instead of __init__.
This is now true in SVN. Previously, __array_finalize__ was not called if the "parent" was NULL. However, now, it is still called with None as the value of the first argument.
Thus __array_finalize__ will be called whenever ndarray.__new__(<some subclass>,...) is called.
Why this change in style from the the common Python idom of __new__, __init__, with the same signature to __new__, __array_finalize__ with possibly different signatures?
I don't see it as a change in style but adding a capability to the ndarray subclass. The problem is that arrays can be created in many ways (slicing, ufuncs, etc). Not all of these ways should go through the __new__/__init__ -- style creation mechanism. Try inheriting from a float builtin and add attributes. Then add your float to an instance of your new class and see what happens. You will get a float-type on the output. This is the essence of Paul's insight that sub-classing is rarely useful because you end up having to re-define all the operators anyway to return the value that you want. He knows whereof he speaks as well, because he wrote MA and UserArray and had experience with Python sub-classing. I wanted a mechanism to make it easier to sub-class arrays and have the operators return your object if possible (including all of it's attributes). Thus, __array_priority__ (a floating point attribute) __array_finalize__ (a method called on internal construction of the array wrapper). were invented (along with __array_wrap__ which any class can define to have their objects survive ufuncs). It was easy enough to see where to call __array_finalize__ in the C-code if somewhat difficult to explain (and get exception handling to work because of my initial over-thinking). The signature is just __array_finalize__(self, parent): return i.e. any return value is ignored (but exceptions are caught). I've used the feature succesfully on at least 3-subclasses (chararray, memmap, and matrix) and so I'm actually pretty happy with it. __new__ and __init__ are still relevant for constructing your brand-new object. The __array_finalize__ function is just what the internal contructor that acutally allocates memory will always call to let you set final attributes *every* time your sub-class gets created. -Travis
__array_priority__ (a floating point attribute) __array_wrap__ (which any class can define to have their objects survive ufuncs)
What do these do again? That is, how are they used internally? What happens if they're not used? If I (or anyone else) is to put together a wiki page about this (see the other thread I just emailed to, please), getting good concise descriptions of what ndarray subclasses need to do/can do would be very helpful. Zach
Travis Oliphant wrote:
Travis Oliphant wrote:
Stefan van der Walt wrote:
The __init__ and __new__ methods are not called because they may have arbitrary signatures. Instead, the __array_finalize__ method is always called. So, you should use that instead of __init__.
This is now true in SVN. Previously, __array_finalize__ was not called if the "parent" was NULL. However, now, it is still called with None as the value of the first argument.
Thus __array_finalize__ will be called whenever ndarray.__new__(<some subclass>,...) is called.
Why this change in style from the the common Python idom of __new__, __init__, with the same signature to __new__, __array_finalize__ with possibly different signatures?
I don't see it as a change in style but adding a capability to the ndarray subclass. The problem is that arrays can be created in many ways (slicing, ufuncs, etc). Not all of these ways should go through the __new__/__init__ -- style creation mechanism. Try inheriting from a float builtin and add attributes. Then add your float to an instance of your new class and see what happens.
Yes, I've tried this with ndarray - it didn't work. Later, I realized that it wasn't a good thing to try. Colin W.
You will get a float-type on the output. This is the essence of Paul's insight that sub-classing is rarely useful because you end up having to re-define all the operators anyway to return the value that you want. He knows whereof he speaks as well, because he wrote MA and UserArray and had experience with Python sub-classing. I wanted a mechanism to make it easier to sub-class arrays and have the operators return your object if possible (including all of it's attributes).
Thus,
__array_priority__ (a floating point attribute) __array_finalize__ (a method called on internal construction of the array wrapper).
were invented (along with __array_wrap__ which any class can define to have their objects survive ufuncs). It was easy enough to see where to call __array_finalize__ in the C-code if somewhat difficult to explain (and get exception handling to work because of my initial over-thinking). The signature is just
__array_finalize__(self, parent): return
i.e. any return value is ignored (but exceptions are caught).
I've used the feature succesfully on at least 3-subclasses (chararray, memmap, and matrix) and so I'm actually pretty happy with it. __new__ and __init__ are still relevant for constructing your brand-new object. The __array_finalize__ function is just what the internal contructor that acutally allocates memory will always call to let you set final attributes *every* time your sub-class gets created.
-Travis
Colin J. Williams wrote:
I have a subclass Bar, a 1-dim array which has some methods and some attributes. One of the attributes is a view of the Bar to permit different shaping.
The ndarray handles sub-classing a little-bit differently. All array constructors go through the same C-code call which can create sub-classes as well. (it's what gets called by ndarray.__new__). If there is a parent object, then additionally, the __array_finalize__(self, parent) method is called right-after creation of the sub-class. This is where attributes should be finalized. But, care must be taken in this code so that a recursion is not setup If this mechanism is not sufficient for you, then you need to use a container class (for this reason UserArray has been resurrected to serve as a default container class model---it needs more testing, however). The problem __array_finalize__ helps fix is how to get subclasses to work well without having to re-define every single special method like UserArray does. For the most part it seems to work, but I suppose it creates a few surprises if you are not aware of what is going on. The most important thing to remember is that attributes are not automatically carried over to new instances because new instances can be created without every calling __new__ or __init__. I'm sure this mechanism can be improved upon and I welcome suggestions.
Suppose that 'a' is an instance of 'Bar', which has a method 'show' and a view attribute 'v'. a ^ 15 returns a Bar instance, with its methods but without the attributes.
I am attempt to change this, Bar has a method __xor__, see below:
def __xor__(self, other): ''' Exclusive or: __xor__(x, y) => x ^ y . ''' z=
1 << this loops to the recursion limit result= ArrayType.__xor__(self, other) n= self.n result.n= n result.rowSize= self.rowSize result.show= self.show result.v= _n.reshape(result.view(), (n*n, n*n)) return result
Look at the __array_finalize__ method in defmatrix.py for ideas about how it can be used. -Travis
Travis Oliphant wrote:
Colin J. Williams wrote:
I have a subclass Bar, a 1-dim array which has some methods and some attributes. One of the attributes is a view of the Bar to permit different shaping.
The ndarray handles sub-classing a little-bit differently. All array constructors go through the same C-code call which can create sub-classes as well. (it's what gets called by ndarray.__new__).
If there is a parent object, then additionally, the
__array_finalize__(self, parent)
method is called right-after creation of the sub-class. This is where attributes should be finalized. But, care must be taken in this code so that a recursion is not setup
If this mechanism is not sufficient for you, then you need to use a container class (for this reason UserArray has been resurrected to serve as a default container class model---it needs more testing, however). The problem __array_finalize__ helps fix is how to get subclasses to work well without having to re-define every single special method like UserArray does.
For the most part it seems to work, but I suppose it creates a few surprises if you are not aware of what is going on.
The most important thing to remember is that attributes are not automatically carried over to new instances because new instances can be created without every calling __new__ or __init__.
I'm sure this mechanism can be improved upon and I welcome suggestions.
Thanks for this. Does this mean that whenever we subclass ndarray, we should use __array_finalize__ (with its additional 'parent' parameter) instead of Python's usual __init__? It would help if you could clarify the role of 'parent'. [Dbg]>>> h(self.__array_finalize__) Help on built-in function __array_finalize__: __array_finalize__(...) Is parent the next type up in the type hierarchy? If so, can this not be determined from self.__class__? I've tried a similar operation with the Python library's sets.Set. There, __init__ is called, ensuring that the expression is of the appropriate sub-type.
Suppose that 'a' is an instance of 'Bar', which has a method 'show' and a view attribute 'v'. a ^ 15 returns a Bar instance, with its methods but without the attributes.
I am attempt to change this, Bar has a method __xor__, see below:
def __xor__(self, other): ''' Exclusive or: __xor__(x, y) => x ^ y . ''' z=
1 << this loops to the recursion limit result= ArrayType.__xor__(self, other) n= self.n result.n= n result.rowSize= self.rowSize result.show= self.show result.v= _n.reshape(result.view(), (n*n, n*n)) return result
Look at the __array_finalize__ method in defmatrix.py for ideas about how it can be used.
def __array_finalize__(self, obj): ndim = self.ndim if ndim == 0: self.shape = (1,1) elif ndim == 1: self.shape = (1,self.shape[0]) return These are functions for which one would use __init__ in numarray. This doesn't describe or illustrate the role or purpose of the parent object. Colin W.
participants (5)
-
Colin J. Williams -
Stefan van der Walt -
Travis Oliphant -
Travis Oliphant -
Zachary Pincus