truthiness of object arrays
I am puzzled by the following (numpy 1.9.0, python 3.4.2): In [1]: t = array(None); t[()] = array([None, None]) # Construct a 0d array of dtype object, containing a single numpy array with 2 elements In [2]: bool(t) Out[2]: True In [3]: if t: pass --------------------------------------------------------------------------- ValueError Traceback (most recent call last) ValueError: The truth value of an array with more than one element is ambiguous. Use a.any() or a.all() I thought that "if x" simply calls "bool", but apparently this is not even the case... Antony
Hi Antony, In general, you can't use numpy arrays in if statements without using any() or all() or some other means of obtaining a single boolean value from the whole array. I think your confusion is that bool() uses truth testing rules outlined here: https://docs.python.org/release/2.5.2/lib/truth.html If statements in theory have equivalent behavior, but take slightly different paths (they don't call bool() directly). This SO post was enlightening to me: http://stackoverflow.com/questions/11885382/is-there-any-difference-between-... Without looking at Numpy's code, I'd bet Numpy arrays probably define __bool__ or __nonzero__ in such a way that the ValueError is raised when it makes sense to do so. HTH, Mike On Tue, Nov 11, 2014 at 12:46 PM, Antony Lee <antony.lee@berkeley.edu> wrote:
I am puzzled by the following (numpy 1.9.0, python 3.4.2):
In [1]: t = array(None); t[()] = array([None, None]) # Construct a 0d array of dtype object, containing a single numpy array with 2 elements
In [2]: bool(t) Out[2]: True
In [3]: if t: pass --------------------------------------------------------------------------- ValueError Traceback (most recent call last) ValueError: The truth value of an array with more than one element is ambiguous. Use a.any() or a.all()
I thought that "if x" simply calls "bool", but apparently this is not even the case...
Antony
_______________________________________________ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
I know you can't in general, but this was in a context where I knew the array contained a single element, which "works" (it checks the truthiness of the contained element). Of course I didn't consider the case where the element contained was itself a (non-trivial) array, thus the finding. The link you posted doesn't seem to address what magic numpy can do to make "bool(x)" and "if x:" have different behaviors (FWIW, "t.__bool__()" also returns True). Antony 2014-11-12 20:15 GMT-08:00 Michael Sarahan <msarahan@gmail.com>:
Hi Antony,
In general, you can't use numpy arrays in if statements without using any() or all() or some other means of obtaining a single boolean value from the whole array.
I think your confusion is that bool() uses truth testing rules outlined here: https://docs.python.org/release/2.5.2/lib/truth.html
If statements in theory have equivalent behavior, but take slightly different paths (they don't call bool() directly). This SO post was enlightening to me:
http://stackoverflow.com/questions/11885382/is-there-any-difference-between-...
Without looking at Numpy's code, I'd bet Numpy arrays probably define __bool__ or __nonzero__ in such a way that the ValueError is raised when it makes sense to do so.
HTH, Mike
On Tue, Nov 11, 2014 at 12:46 PM, Antony Lee <antony.lee@berkeley.edu> wrote:
I am puzzled by the following (numpy 1.9.0, python 3.4.2):
In [1]: t = array(None); t[()] = array([None, None]) # Construct a 0d array of dtype object, containing a single numpy array with 2 elements
In [2]: bool(t) Out[2]: True
In [3]: if t: pass
--------------------------------------------------------------------------- ValueError Traceback (most recent call last) ValueError: The truth value of an array with more than one element is ambiguous. Use a.any() or a.all()
I thought that "if x" simply calls "bool", but apparently this is not even the case...
Antony
_______________________________________________ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
_______________________________________________ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
On Mi, 2014-11-12 at 22:19 -0800, Antony Lee wrote:
I know you can't in general, but this was in a context where I knew the array contained a single element, which "works" (it checks the truthiness of the contained element). Of course I didn't consider the case where the element contained was itself a (non-trivial) array, thus the finding. The link you posted doesn't seem to address what magic numpy can do to make "bool(x)" and "if x:" have different behaviors (FWIW, "t.__bool__()" also returns True). Antony
First sight, sounds like a bug. It should be passing out the error raised by the array inside. Can you open an issue? - Sebastian
2014-11-12 20:15 GMT-08:00 Michael Sarahan <msarahan@gmail.com>: Hi Antony,
In general, you can't use numpy arrays in if statements without using any() or all() or some other means of obtaining a single boolean value from the whole array.
I think your confusion is that bool() uses truth testing rules outlined here: https://docs.python.org/release/2.5.2/lib/truth.html
If statements in theory have equivalent behavior, but take slightly different paths (they don't call bool() directly). This SO post was enlightening to me: http://stackoverflow.com/questions/11885382/is-there-any-difference-between-...
Without looking at Numpy's code, I'd bet Numpy arrays probably define __bool__ or __nonzero__ in such a way that the ValueError is raised when it makes sense to do so.
HTH, Mike
On Tue, Nov 11, 2014 at 12:46 PM, Antony Lee <antony.lee@berkeley.edu> wrote:
I am puzzled by the following (numpy 1.9.0, python 3.4.2):
In [1]: t = array(None); t[()] = array([None, None]) # Construct a 0d array of dtype object, containing a single numpy array with 2 elements
In [2]: bool(t) Out[2]: True
In [3]: if t: pass --------------------------------------------------------------------------- ValueError Traceback (most recent call last) ValueError: The truth value of an array with more than one element is ambiguous. Use a.any() or a.all()
I thought that "if x" simply calls "bool", but apparently this is not even the case...
Antony
_______________________________________________ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
_______________________________________________ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
_______________________________________________ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
On 11/13/2014 1:19 AM, Antony Lee wrote:
"t.__bool__()" also returns True
But t.__nonzero__() is being called in the `if` test. The question is: is the difference between `__nonzero__` and `__bool__` intentional. By the way, there has been a change in behavior. For example, in 1.7.1 if you call `t.__bool__()` it raised an attribute error -- unless one first called `t.__nonzero__()` and then called `t.__bool__()`, which was of course very weird and needed to be fixed. Maybe (?) not like this. In fact the oddity probably remains but moved. in 1.9.0 I see this:
np.__version__ '1.9.0' t = np.array(None); t[()] = np.array([None, None]) t.__nonzero__() Traceback (most recent call last): File "<stdin>", line 1, in <module> AttributeError: 'numpy.ndarray' object has no attribute '__nonzero__' t.__bool__() True t.__nonzero__() ValueError: The truth value of an array with more than one element is ambiguous. Use a.any() or a.all()
Alan Isaac
On Do, 2014-11-13 at 08:24 -0500, Alan G Isaac wrote:
On 11/13/2014 1:19 AM, Antony Lee wrote:
"t.__bool__()" also returns True
But t.__nonzero__() is being called in the `if` test. The question is: is the difference between `__nonzero__` and `__bool__` intentional.
By the way, there has been a change in behavior. For example, in 1.7.1 if you call `t.__bool__()` it raised an attribute error -- unless one first called `t.__nonzero__()` and then called `t.__bool__()`, which was of course very weird and needed to be fixed. Maybe (?) not like this.
In fact the oddity probably remains but moved. in 1.9.0 I see this:
np.__version__ '1.9.0' t = np.array(None); t[()] = np.array([None, None]) t.__nonzero__() Traceback (most recent call last): File "<stdin>", line 1, in <module> AttributeError: 'numpy.ndarray' object has no attribute '__nonzero__' t.__bool__() True t.__nonzero__() ValueError: The truth value of an array with more than one element is ambiguous. Use a.any() or a.all()
Note that the error is set, but appears to be not indicated within the C-api. This can cause errors to be missed or appear later then they should... Might be just that you are seeing, though I could not find a reason for it in the numpy code on first glance. - Sebastian
Alan Isaac
_______________________________________________ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
On Python3, __nonzero__ is never defined (always raises an AttributeError), even after calling __bool__. 2014-11-13 5:24 GMT-08:00 Alan G Isaac <alan.isaac@gmail.com>:
On 11/13/2014 1:19 AM, Antony Lee wrote:
"t.__bool__()" also returns True
But t.__nonzero__() is being called in the `if` test. The question is: is the difference between `__nonzero__` and `__bool__` intentional.
By the way, there has been a change in behavior. For example, in 1.7.1 if you call `t.__bool__()` it raised an attribute error -- unless one first called `t.__nonzero__()` and then called `t.__bool__()`, which was of course very weird and needed to be fixed. Maybe (?) not like this.
In fact the oddity probably remains but moved. in 1.9.0 I see this:
np.__version__ '1.9.0' t = np.array(None); t[()] = np.array([None, None]) t.__nonzero__() Traceback (most recent call last): File "<stdin>", line 1, in <module> AttributeError: 'numpy.ndarray' object has no attribute '__nonzero__' t.__bool__() True t.__nonzero__() ValueError: The truth value of an array with more than one element is ambiguous. Use a.any() or a.all()
Alan Isaac
_______________________________________________ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
On 11/13/2014 12:37 PM, Antony Lee wrote:
On Python3, __nonzero__ is never defined (always raises an AttributeError), even after calling __bool__.
The example I posted was Python 3.4.1 with numpy 1.9.0. fwiw, Alan Isaac Python 3.4.1 (v3.4.1:c0e311e010fc, May 18 2014, 10:38:22) [MSC v.1600 32 bit (Intel)] on win32 Type "help", "copyright", "credits" or "license" for more information.
import numpy as np np.__version__ '1.9.0' t = np.array(None); t[()] = np.array([None, None]) t.__nonzero__() Traceback (most recent call last): File "<stdin>", line 1, in <module> AttributeError: 'numpy.ndarray' object has no attribute '__nonzero__' t.__bool__() True t.__nonzero__() ValueError: The truth value of an array with more than one element is ambiguous. Use a.any() or a.all()
Dunno, seems unlikely that something changed with Python 3.4.2... $ python --version Python 3.4.2 $ python -c 'import numpy as np; print(np.__version__); t = np.array(None); t[()] = np.array([None, None]); t.__bool__(); t.__nonzero__()' 1.9.0 Traceback (most recent call last): File "<string>", line 1, in <module> AttributeError: 'numpy.ndarray' object has no attribute '__nonzero__' 2014-11-13 10:05 GMT-08:00 Alan G Isaac <alan.isaac@gmail.com>:
On 11/13/2014 12:37 PM, Antony Lee wrote:
On Python3, __nonzero__ is never defined (always raises an AttributeError), even after calling __bool__.
The example I posted was Python 3.4.1 with numpy 1.9.0.
fwiw, Alan Isaac
Python 3.4.1 (v3.4.1:c0e311e010fc, May 18 2014, 10:38:22) [MSC v.1600 32 bit (Intel)] on win32 Type "help", "copyright", "credits" or "license" for more information.
import numpy as np np.__version__ '1.9.0' t = np.array(None); t[()] = np.array([None, None]) t.__nonzero__() Traceback (most recent call last): File "<stdin>", line 1, in <module> AttributeError: 'numpy.ndarray' object has no attribute '__nonzero__' t.__bool__() True t.__nonzero__() ValueError: The truth value of an array with more than one element is ambiguous. Use a.any() or a.all()
_______________________________________________ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
On Thu, Nov 13, 2014 at 1:24 PM, Alan G Isaac <alan.isaac@gmail.com> wrote:
On 11/13/2014 1:19 AM, Antony Lee wrote:
"t.__bool__()" also returns True
But t.__nonzero__() is being called in the `if` test. The question is: is the difference between `__nonzero__` and `__bool__` intentional.
__nonzero__ and __bool__ should be identical; they refer to the same magic method, it's just that Py2 calls it __nonzero__ and Py3 calls it __bool__. bool(t) and 'if t: ...' should always be identical at the interpreter level; that they aren't is probably because we're someone violating some invariant in the Python C API (like @seberg suggested). In Py2 they both call __nonzero__, and in Py3 they both call __bool__.
By the way, there has been a change in behavior. For example, in 1.7.1 if you call `t.__bool__()` it raised an attribute error -- unless one first called `t.__nonzero__()` and then called `t.__bool__()`, which was of course very weird and needed to be fixed. Maybe (?) not like this.
In fact the oddity probably remains but moved. in 1.9.0 I see this:
np.__version__ '1.9.0' t = np.array(None); t[()] = np.array([None, None]) t.__nonzero__() Traceback (most recent call last): File "<stdin>", line 1, in <module> AttributeError: 'numpy.ndarray' object has no attribute '__nonzero__' t.__bool__() True t.__nonzero__() ValueError: The truth value of an array with more than one element is ambiguous. Use a.any() or a.all()
I think you're being misled by buggy exception handling weirdness, where the ValueError raised by calling __bool__ is getting delayed, and then pre-empting the AttributeError that should be generated by the call to __nonzero__. Consider:
t.__bool__() True 1 # any expression here will do ValueError: The truth value of an array with more than one element is ambiguous. Use a.any() or a.all() t.__nonzero__() Traceback (most recent call last): File "<stdin>", line 1, in <module> AttributeError: 'numpy.ndarray' object has no attribute '__nonzero__'
-n -- Nathaniel J. Smith Postdoctoral researcher - Informatics - University of Edinburgh http://vorpus.org
On 11/13/2014 1:32 PM, Nathaniel Smith wrote:
I think you're being misled by buggy exception handling weirdness, where the ValueError raised by calling __bool__ is getting delayed, and then pre-empting the AttributeError that should be generated by the call to __nonzero__.
Aha! Thanks.
On Thu, Nov 13, 2014 at 8:24 AM, Alan G Isaac <alan.isaac@gmail.com> wrote:
On 11/13/2014 1:19 AM, Antony Lee wrote:
"t.__bool__()" also returns True
But t.__nonzero__() is being called in the `if` test. The question is: is the difference between `__nonzero__` and `__bool__` intentional.
By the way, there has been a change in behavior. For example, in 1.7.1 if you call `t.__bool__()` it raised an attribute error -- unless one first called `t.__nonzero__()` and then called `t.__bool__()`, which was of course very weird and needed to be fixed. Maybe (?) not like this.
The change probably came in with the Python2/3 common code base. Python3 replaces `__nonzero__` with `__bool__`. IIRC, the two are now the same method. I don't have the code available to check... <snip> Chuck
participants (6)
-
Alan G Isaac -
Antony Lee -
Charles R Harris -
Michael Sarahan -
Nathaniel Smith -
Sebastian Berg