Dtype objects throw an exception if compared for equality against other objects. e.g.
import numpy numpy.dtype('uint32')==1 Traceback (most recent call last): File "<stdin>", line 1, in <module> TypeError: data type not understood
After some googling, I think python wisdom (given in the Python docs) says that equality testing should never throw an exception, but return the NotImplemented singleton where the comparison does not make "sense". Throwing an exception breaks membership testing for lists/tuples (using the 'in' operator calls implicitly calls __eq__()). My real-world example is serialisation of numpy arrays using PyYAML. This module should serialise any picklable object including ndarrays. However, it performs a test on each object: if data in [None, ()]: blah... This fails on the dtype object of the array. What's the consensus on this? Is the current dtype behaviour broken? cheers, BC
On Sun, Jan 11, 2009 at 03:41, Bryan Cole <bryan@cole.uklinux.net> wrote:
Dtype objects throw an exception if compared for equality against other objects. e.g.
import numpy numpy.dtype('uint32')==1 Traceback (most recent call last): File "<stdin>", line 1, in <module> TypeError: data type not understood
After some googling, I think python wisdom (given in the Python docs) says that equality testing should never throw an exception, but return the NotImplemented singleton where the comparison does not make "sense". Throwing an exception breaks membership testing for lists/tuples (using the 'in' operator calls implicitly calls __eq__()).
My real-world example is serialisation of numpy arrays using PyYAML. This module should serialise any picklable object including ndarrays. However, it performs a test on each object:
if data in [None, ()]: blah...
This fails on the dtype object of the array.
What's the consensus on this? Is the current dtype behaviour broken?
It's suboptimal, certainly. Feel free to fix it. However, also note that with ndarray's rich comparisons, such membership testing will fail with ndarrays, too. -- Robert Kern "I have come to believe that the whole world is an enigma, a harmless enigma that is made terrible by our own mad attempt to interpret it as though it had an underlying truth." -- Umberto Eco
What's the consensus on this? Is the current dtype behaviour broken?
It's suboptimal, certainly. Feel free to fix it.
Thankyou.
However, also note that with ndarray's rich comparisons, such membership testing will fail with ndarrays, too.
This poses a similarly big problem. I can't understand this behaviour either:
a = numpy.array([1.2,3.4]) a == 6.5 array([False, False], dtype=bool) a == numpy.array([1,2]) array([False, False], dtype=bool) a == "foo" False a in [1,2,3] Traceback (most recent call last): File "<stdin>", line 1, in <module> ValueError: The truth value of an array with more than one element is ambiguous. Use a.any() or a.all()
Surely this is a bug?! This membership testing operation is quite reasonable. If list.__contains__ is calling ndarray.__eq__(), this ought to succeed. Why does this fail? (and I fail to see how any() or all() can resolve the situation). I need to go check what list.__contains__() is doing... BC
However, also note that with ndarray's rich comparisons, such membership testing will fail with ndarrays, too.
This poses a similarly big problem. I can't understand this behaviour either:
OK, I can now. After equality testing each item, the result must be cast to bool. This is where the truth-testing comes in. Testing array membership in lists does work, just so long as there are no numerical objects in the list. Any/only equality comparisons which return another (bool) ndarray will cause the exception. Thus, the error is not triggered in PyYAML. FWIW, I don't like the policy of forbidding truth testing on bare arrays. I'd prefer that ndarray.__nonzero__() implicitly call ndarray.all(). For my use-cases, this is what I want >90% of the time. Where any() is required, this must be made explicit. However, I understand the logic behind raising an exception and presumably others prefer this. Ultimately, I'm forced to conclude that the idea that membership testing should always succeed was bogus. Bummer. BC
On Sun, Jan 11, 2009 at 15:13, Bryan Cole <bryan@cole.uklinux.net> wrote:
However, also note that with ndarray's rich comparisons, such membership testing will fail with ndarrays, too.
This poses a similarly big problem. I can't understand this behaviour either:
OK, I can now. After equality testing each item, the result must be cast to bool. This is where the truth-testing comes in.
Testing array membership in lists does work, just so long as there are no numerical objects in the list. Any/only equality comparisons which return another (bool) ndarray will cause the exception. Thus, the error is not triggered in PyYAML.
FWIW, I don't like the policy of forbidding truth testing on bare arrays. I'd prefer that ndarray.__nonzero__() implicitly call ndarray.all(). For my use-cases, this is what I want >90% of the time. Where any() is required, this must be made explicit. However, I understand the logic behind raising an exception and presumably others prefer this.
That almost works for == but doesn't for !=. We used to use .any() in Numeric (if any element is nonzero, then the array is nonzero). Many people were confused. There was a lot of broken code out there. Moving to numpy, we decided to prevent that kind of silently broken code. In any case, it still wouldn't work. Because the argument are broadcasted against each other, you can have two differently shaped arrays compare equal. E.g. (ones([5,3]) == 1.0).all() == True.
Ultimately, I'm forced to conclude that the idea that membership testing should always succeed was bogus. Bummer.
Yup. Sorry. It's an unfortunate effect of allowing rich comparisons. -- Robert Kern "I have come to believe that the whole world is an enigma, a harmless enigma that is made terrible by our own mad attempt to interpret it as though it had an underlying truth." -- Umberto Eco
participants (2)
-
Bryan Cole -
Robert Kern