Overload binary operators with custom array type
I have written my own array class, let's call it "myarray", which behaves like a numpy.ndarray. I would like to support binary operators with mixed operand types, i.e.: lhs + rhs where one of the operands is a myarray object, and the other is a numpy.ndarray object. The problem is that I want the operation in this case to be handled by my add method, not by numpy's add method. If lhs is a myarray object, then this is no problem, since myarray.__add__(lhs, rhs) will be called. However if lhs is a numpy array and rhs is a myarray, then numpy.ndarray.__add__(lhs, rhs) will be called, whereas what I would like to happen would be for myarray.__radd__(rhs, lhs) to be called. Is there a way I can get numpy to do the latter? Python will only call the rhs.__radd__ method if lhs.__add__ returns NotImplemented, but as far as I can tell, the numpy.ndarray.__add__ method never returns NotImplemented. It seems that for "lhs + rhs", where lhs is a numpy array, numpy does one of two things: 1. if rhs has an __array__ method, than it calls rsh.__array__(), which is expected to return a numpy.ndarray object. Then it performs the binary operation with the result of __array__ for the rhs. 2. If rhs doesn't have an __array__ method, then it treats rhs as some kind of scalar, and tries to add each element of lhs to rhs. Under no circumstances does numpy just assume that it doesn't know what to do with rhs and return NotImplemented. My question is, is there a way I can get the effect of having myarray.__radd__(rhs, lhs) in this case, or in some way get numpy to invoke my code to handle this operation rather than just assuming rhs is a scalar? As for why I don't just implement myarray.__array__(), the reason is because it would be expensive. A myarray object doesn't actually contain the array data; rather it contains a handle referring to an array which resides in another process. The server process which actually contains the arrays can even be on another machine. So converting a myarray object to a numpy.ndarray involves transferring all the array data to the python process. For various reasons, for my application it's preferable that I handle binary operators with mixed numpy/myarray operand types myself, rather than converting the myarray object to a numpy.ndarray. Thanks in advance for any suggestions you can give. Adam
On 1/12/07, Adam Jenkins <adam@thejenkins.org> wrote:
I have written my own array class, let's call it "myarray", which behaves like a numpy.ndarray. I would like to support binary operators with mixed operand types, i.e.:
lhs + rhs
where one of the operands is a myarray object, and the other is a numpy.ndarray object.
The problem is that I want the operation in this case to be handled by my add method, not by numpy's add method. If lhs is a myarray object, then this is no problem, since myarray.__add__(lhs, rhs) will be called. However if lhs is a numpy array and rhs is a myarray, then numpy.ndarray.__add__(lhs, rhs) will be called, whereas what I would like to happen would be for myarray.__radd__(rhs, lhs) to be called. Is there a way I can get numpy to do the latter? Python will only call the rhs.__radd__ method if lhs.__add__ returns NotImplemented, but as far as I can tell, the numpy.ndarray.__add__ method never returns NotImplemented.
It seems that for "lhs + rhs", where lhs is a numpy array, numpy does one of two things:
1. if rhs has an __array__ method, than it calls rsh.__array__(), which is expected to return a numpy.ndarray object. Then it performs the binary operation with the result of __array__ for the rhs.
2. If rhs doesn't have an __array__ method, then it treats rhs as some kind of scalar, and tries to add each element of lhs to rhs.
Under no circumstances does numpy just assume that it doesn't know what to do with rhs and return NotImplemented. My question is, is there a way I can get the effect of having myarray.__radd__(rhs, lhs) in this case, or in some way get numpy to invoke my code to handle this operation rather than just assuming rhs is a scalar?
As for why I don't just implement myarray.__array__(), the reason is because it would be expensive. A myarray object doesn't actually contain the array data; rather it contains a handle referring to an array which resides in another process. The server process which actually contains the arrays can even be on another machine. So converting a myarray object to a numpy.ndarray involves transferring all the array data to the python process. For various reasons, for my application it's preferable that I handle binary operators with mixed numpy/myarray operand types myself, rather than converting the myarray object to a numpy.ndarray.
You want __array_priority__; add an attribute __array_priority__ to your array-like object and set it's value to, say, 10 and ndarrays will defer to your object. -- //=][=\\ tim.hochberg@ieee.org
Timothy Hochberg <tim.hochberg <at> ieee.org> writes:
You want __array_priority__; add an attribute __array_priority__ to your array-like object and set it's value to, say, 10 and ndarrays will defer to your object.
Thank you very much. I had seen the __array_priority__ property mentioned in the Numpybook, but it's only documented there as being used to decide which operand's type the result of an operation should be converted to, so I didn't realize it would also be used to decide which operand to defer the operation to. I just tried it, and it does solve the problem for the mathematical operators like + and -, but doesn't seem to work for the comparison operators. That is, if I write lhs < rhs where lhs is a numpy.ndarray and rhs is a myarray, numpy still treats rhs as a scalar and tries to compare each element of lhs to rhs, even though I set myarray.__array_priority__. Is this intentional that the comparison operators ignore __array_priority__, or just a bug in ndarray? Thanks again. Adam
Adam Jenkins wrote:
Timothy Hochberg <tim.hochberg <at> ieee.org> writes:
You want __array_priority__; add an attribute __array_priority__ to your
array-like object and set it's value to, say, 10 and ndarrays will defer to your object.
Thank you very much. I had seen the __array_priority__ property mentioned in the Numpybook, but it's only documented there as being used to decide which operand's type the result of an operation should be converted to, so I didn't realize it would also be used to decide which operand to defer the operation to.
I just tried it, and it does solve the problem for the mathematical operators like + and -, but doesn't seem to work for the comparison operators. That is, if I write
lhs < rhs
where lhs is a numpy.ndarray and rhs is a myarray, numpy still treats rhs as a scalar and tries to compare each element of lhs to rhs, even though I set myarray.__array_priority__. Is this intentional that the comparison operators ignore __array_priority__, or just a bug in ndarray?
Probably could be considered a bug. What does less(lhs, rhs) do? -Travis
Travis Oliphant <oliphant.travis <at> ieee.org> writes:
Adam Jenkins wrote:
I just tried it, and it does solve the problem for the mathematical operators like + and -, but doesn't seem to work for the comparison operators. That is, if I write
lhs < rhs
where lhs is a numpy.ndarray and rhs is a myarray, numpy still treats rhs as a scalar and tries to compare each element of lhs to rhs, even though I set myarray.__array_priority__. Is this intentional that the comparison operators ignore __array_priority__, or just a bug in ndarray?
Probably could be considered a bug.
What does
less(lhs, rhs)
do?
It does the same thing, calls rhs.__gt__ on each element of lhs. Hmm, I just tried numpy.add(lhs, rhs), and that returns NotImplemented, whereas "lhs+rhs" results in rhs.__radd__(lhs) being called. So "lhs<rhs" behaves the same as numpy.less(lhs,rhs), but "lhs+rhs" and numpy.add(lhs,rhs) behave differently. Here's a sample program which demonstrates all this. I'm using python 2.4.3 and numpy 1.0.1. Adam Jenkins test.py ----------- import numpy class myarray(object): def _get_array_priority(self): print "myarray.__array_priority__" return 20 __array_priority__ = property(_get_array_priority) def __add__(self, other): print "myarray.__add__(%s)" % (other,) return 1 def __radd__(self, other): print "myarray.__radd__(%s)" % (other,) return 1 def __lt__(self, other): print "myarray.__lt__(%s)" % (other,) return True def __gt__(self, other): print "myarray.__gt__(%s)" % (other,) return True lhs = numpy.array([1,2,3]) rhs = myarray() print "lhs+rhs=%s\n" % (lhs + rhs,) print "add(lhs, rhs)=%s\n" % (numpy.add(lhs, rhs),) print "lhs<rhs=%s\n" % (lhs < rhs,) print "less(lhs, rhs)=%s\n" % (numpy.less(lhs, rhs),)
participants (3)
-
Adam Jenkins -
Timothy Hochberg -
Travis Oliphant