Why is the truth value of ndarray not simply size>0 ?
Is there a reason why ndarray truth tests (except scalars) deviates from the convention of other Python iterables list,array.array,str,dict,... ? Furthermore there is a surprising strange exception for arrays with size 1 (!= scalars). I often run into exceptions and unexpected bahavior like shown below, when developing or transcribing from other types to numpy. Robert ---
a=np.array([12,4,5]) if a: print 2 ... Traceback (most recent call last): File "<interactive input>", line 1, in <module> ValueError: The truth value of an array with more than one element is ambiguous. Use a.any() or a.all() a=np.array([12]) if a: print 2 ... 2 a=np.array([0]) if a: print 2 ... a=[0] if a: print 2 ... 2 a=[] if a: print 2 ... import array a=array.array('i',[12,1,3]) if a: print 2 ... 2 a=array.array('i',[0]) if a: print 2 ... 2 a=array.array('i',[]) if a: print 2 ... bool(np.array(0)) False
On 2009-09-07 07:11 , Robert wrote:
Is there a reason why ndarray truth tests (except scalars) deviates from the convention of other Python iterables list,array.array,str,dict,... ?
Furthermore there is a surprising strange exception for arrays with size 1 (!= scalars).
Historically, numpy's predecessors used "not equal to zero" as the meaning for truth (consistent with numerical types in Python). However, this introduces an ambiguity as both any(a != 0) and all(a != 0) are reasonable interpretations of the truth value of a sequence of numbers. Numpy refuses to guess and raises the exception shown below. For sequences with a single item, there is no ambiguity and numpy does the (numerically) ordinary thing. The ndarray type available in Numpy is not conceptually an extension of Python's iterables. If you'd like to help other Numpy users with this issue, you can edit the documentation in the online documentation editor at http://docs.scipy.org/numpy/docs/numpy-docs/user/index.rst -Neil
Neil Martinsen-Burrell wrote:
On 2009-09-07 07:11 , Robert wrote:
Is there a reason why ndarray truth tests (except scalars) deviates from the convention of other Python iterables list,array.array,str,dict,... ?
Furthermore there is a surprising strange exception for arrays with size 1 (!= scalars).
Historically, numpy's predecessors used "not equal to zero" as the meaning for truth (consistent with numerical types in Python). However, this introduces an ambiguity as both any(a != 0) and all(a != 0) are reasonable interpretations of the truth value of a sequence of numbers.
well, I can familiarize with that "not equal to zero" philosophy for a math-centric array type (different from a container / size>0 philosophy) However I don't see that all(a) (or "all(a != 0)") is something which anybody would ever expect with .__nonzero__() / if a: ... . Does anybody? And the current behavior with all those strange exceptions and exceptions from exceptions still seems awkward and unnecessary. The any() interpretion is outstandingly "right" in my opinion, and doesn't need to be guessed: anything/any part non-zero disturbs the clean "zeroness". Zero must be wholly pure zero. This is so everywhere in math and informatics. a number/memory is zero when all bits/bytes are zero. a matrix is a zero matrix when all elements are zero... This way only the test is also seamlessly consistent with a zero length array (while all(zerolengtharray != 0) would be True surprisingly!) This kind of any(a) truth test (only) is also often needed, and it would be also fast executable this way. It would be compatible with None/False init/default variable tests during code evolution in Python style and would behave well everywhere as far as I can see. It would also not break old code. Would a feature request in that direction have any chance? Robert
Numpy refuses to guess and raises the exception shown below. For sequences with a single item, there is no ambiguity and numpy does the (numerically) ordinary thing.
The ndarray type available in Numpy is not conceptually an extension of Python's iterables. If you'd like to help other Numpy users with this issue, you can edit the documentation in the online documentation editor at http://docs.scipy.org/numpy/docs/numpy-docs/user/index.rst
-Neil
Robert wrote:
Neil Martinsen-Burrell wrote:
Is there a reason why ndarray truth tests (except scalars) deviates from the convention of other Python iterables list,array.array,str,dict,... ?
Furthermore there is a surprising strange exception for arrays with size 1 (!= scalars). Historically, numpy's predecessors used "not equal to zero" as the meaning for truth (consistent with numerical types in Python). However,
On 2009-09-07 07:11 , Robert wrote: this introduces an ambiguity as both any(a != 0) and all(a != 0) are reasonable interpretations of the truth value of a sequence of numbers.
well, I can familiarize with that "not equal to zero" philosophy for a math-centric array type (different from a container / size>0 philosophy)
However I don't see that all(a) (or "all(a != 0)") is something which anybody would ever expect with .__nonzero__() / if a: ... . Does anybody? And the current behavior with all those strange exceptions and exceptions from exceptions still seems awkward and unnecessary.
The any() interpretion is outstandingly "right" in my opinion, and doesn't need to be guessed: anything/any part non-zero disturbs the clean "zeroness". Zero must be wholly pure zero. This is so everywhere in math and informatics. a number/memory is zero when all bits/bytes are zero. a matrix is a zero matrix when all elements are zero... This way only the test is also seamlessly consistent with a zero length array (while all(zerolengtharray != 0) would be True surprisingly!) This kind of any(a) truth test (only) is also often needed, and it would be also fast executable this way. It would be compatible with None/False init/default variable tests during code evolution in Python style and would behave well everywhere as far as I can see. It would also not break old code. Would a feature request in that direction have any chance?
Robert
Numpy refuses to guess and raises the exception shown below. For sequences with a single item, there is no ambiguity and numpy does the (numerically) ordinary thing.
coming to mind another way to see it: I'm not aware of any other python type which doesn't definitely know if it is __nonzero__ or not (unless there is an IOError or so). And everywhere: if there is *any* logical doubt at all, the default is: True! - not an Exception. For example .__nonzero__/.__bool__ for a custom class defaults to True. A behavior where an object throws an exception upon __nonzero__ test just because of principal doubts seems not to fit into the Python world. The non-zero test must definitely go through. Only 2 ways seem to be consistently Pythonic and logical: "size > 0"; or "any(a)" (*); and the later option may be more 'numerical'. Robert * .__nonzero__() and perhaps .any() too should not fail upon flexible types like currently:
np.array(["","",""]).any() Traceback (most recent call last): File "<interactive input>", line 1, in <module> TypeError: cannot perform reduce with flexible type
On 9/13/2009 7:46 AM, Robert wrote:
2 ways seem to be consistently Pythonic and logical: "size> 0"; or "any(a)" (*); and the later option may be more 'numerical'.
Well, *there's* the problem. As a user I have felt more than once that a length based test, like other containers, would be natural, so I that I could do the usual test if a: That would certainly prove problematic for the `any` test when a is an array of zeros. And then soon after I want to use if (a>0): and that would certainly prove problematic for the `len` test when (a>0) is an array of zeros. And as for the `any` test, well in this case I usually want `all`. So the ValueError has proved useful. Alan Isaac
On Sun, Sep 13, 2009 at 9:05 AM, Alan G Isaac <aisaac@american.edu> wrote:
On 9/13/2009 7:46 AM, Robert wrote:
2 ways seem to be consistently Pythonic and logical: "size> 0"; or "any(a)" (*); and the later option may be more 'numerical'.
Well, *there's* the problem.
As a user I have felt more than once that a length based test, like other containers, would be natural, so I that I could do the usual test if a: That would certainly prove problematic for the `any` test when a is an array of zeros. And then soon after I want to use if (a>0): and that would certainly prove problematic for the `len` test when (a>0) is an array of zeros. And as for the `any` test, well in this case I usually want `all`.
So the ValueError has proved useful.
Alan Isaac _______________________________________________ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
In a numerical context, I essentially stopped using if a: ... I don't think a single definition is common enough to avoid an ambiguous interpretation, and the ValueError is a good reminder to be more precise. I don't see why zero should evaluate as False, except for booleans, even for scalars and also in python. zeros are useful numbers and throwing them together with None is pretty confusing. But I agree with the initial post that for single numbers the numpy behavior is not completely consistent.
a=np.array(0) if not a: print 'not true'
not true
a=np.array([0]) if not a: print 'not true'
not true
a=np.array([]) if not a: print 'not true'
not true
np.array([0]).shape (1,) np.array(0).shape () np.array([]).shape (0,)
I don't think np.array([0]) should evaluate to false. But "if a.size==0:" or "if a==0:" or if np.any(a==0):" is much clearer. Josef
On Sun, Sep 13, 2009 at 05:30, Robert <kxroberto@googlemail.com> wrote:
Neil Martinsen-Burrell wrote:
On 2009-09-07 07:11 , Robert wrote:
Is there a reason why ndarray truth tests (except scalars) deviates from the convention of other Python iterables list,array.array,str,dict,... ?
Furthermore there is a surprising strange exception for arrays with size 1 (!= scalars).
Historically, numpy's predecessors used "not equal to zero" as the meaning for truth (consistent with numerical types in Python). However, this introduces an ambiguity as both any(a != 0) and all(a != 0) are reasonable interpretations of the truth value of a sequence of numbers.
well, I can familiarize with that "not equal to zero" philosophy for a math-centric array type (different from a container / size>0 philosophy)
However I don't see that all(a) (or "all(a != 0)") is something which anybody would ever expect with .__nonzero__() / if a: ... . Does anybody? And the current behavior with all those strange exceptions and exceptions from exceptions still seems awkward and unnecessary.
The any() interpretion is outstandingly "right" in my opinion, and doesn't need to be guessed: anything/any part non-zero disturbs the clean "zeroness". Zero must be wholly pure zero. This is so everywhere in math and informatics. a number/memory is zero when all bits/bytes are zero. a matrix is a zero matrix when all elements are zero... This way only the test is also seamlessly consistent with a zero length array (while all(zerolengtharray != 0) would be True surprisingly!) This kind of any(a) truth test (only) is also often needed, and it would be also fast executable this way. It would be compatible with None/False init/default variable tests during code evolution in Python style and would behave well everywhere as far as I can see. It would also not break old code. Would a feature request in that direction have any chance?
No. Numeric used to use the any() interpretation, and it led to many, many errors in people's code that went undetected for years. For example, people seem to usually want "a == b" to be True iff *all* elements are equal. People also seem to usually want "a != b" to be True if *any* elements are unequal. These desires are inconsistent and cannot be realized at the same time, yet people seem to hold both mental models in their head without thoroughly thinking through the logic or testing it. No amount of documentation or education seemed to help, so we decided to raise an exception instead. -- Robert Kern "I have come to believe that the whole world is an enigma, a harmless enigma that is made terrible by our own mad attempt to interpret it as though it had an underlying truth." -- Umberto Eco
participants (5)
-
Alan G Isaac -
josef.pktd＠gmail.com -
Neil Martinsen-Burrell -
Robert -
Robert Kern