preferred way of testing empty arrays
I have been using numpy for several years and I am very impressed with its flexibility. However, there is one problem that has always bothered me. Quite often I need to test consistently whether a variable is any of the following: an empty list, an empty array or None. Since both arrays and lists are ordered sequences I usually allow for both, and convert if necessary. However, when the (optional) argument is an empty list/array or None, I skip its processing and do nothing. Now, how should I test for 'emptiness'? PEP8 recommends: For sequences, (strings, lists, tuples), use the fact that empty sequences are false.
seq = [] if not seq: ... print 'Hello'
It works for empty numpy arrays:
a = np.array(seq) if not a: ... print 'Hello" Hello
but if 'a' is non-empty it raises an exception:
a = np.array([1,2]) if not a: ... print 'Hello" ValueError: The truth value of an array with more than one element is ambiguous. Use a.any() or a.all()
One solution is to test lengths:
if len(seq) > 0: .... ... if len(a) > 0: ... ...
but for None it fails again:
opt = None if len(opt): ... TypeError: object of type 'NoneType' has no len()
even worse we can not test for None, because it will fail if someone accidentally wraps None in an array:
a = np.array(opt) if opt is not None: ... print 'hello' hello
Although this behaviour is expected, it may be very confusing and it easily leads to errors. Even worse it adds unnecessary complexity in the code, because arrays, lists and None have to be handled differently. I hoped the I managed to explain the problem well. Is there a recommended way to test for empty arrays? Cheers, Bartosz
On Fri, Jan 27, 2012 at 2:46 PM, Bartosz Telenczuk < b.telenczuk@biologie.hu-berlin.de> wrote:
I have been using numpy for several years and I am very impressed with its flexibility. However, there is one problem that has always bothered me.
Quite often I need to test consistently whether a variable is any of the following: an empty list, an empty array or None. Since both arrays and lists are ordered sequences I usually allow for both, and convert if necessary. However, when the (optional) argument is an empty list/array or None, I skip its processing and do nothing.
Now, how should I test for 'emptiness'?
PEP8 recommends:
For sequences, (strings, lists, tuples), use the fact that empty sequences are false.
seq = [] if not seq: ... print 'Hello'
It works for empty numpy arrays:
a = np.array(seq) if not a: ... print 'Hello" Hello
but if 'a' is non-empty it raises an exception:
a = np.array([1,2]) if not a: ... print 'Hello" ValueError: The truth value of an array with more than one element is ambiguous. Use a.any() or a.all()
One solution is to test lengths:
if len(seq) > 0: .... ... if len(a) > 0: ... ...
but for None it fails again:
opt = None if len(opt): ... TypeError: object of type 'NoneType' has no len()
even worse we can not test for None, because it will fail if someone accidentally wraps None in an array:
a = np.array(opt) if opt is not None: ... print 'hello' hello
Although this behaviour is expected, it may be very confusing and it easily leads to errors. Even worse it adds unnecessary complexity in the code, because arrays, lists and None have to be handled differently.
I hoped the I managed to explain the problem well. Is there a recommended way to test for empty arrays?
Cheers,
Bartosz
Don't know if it is recommended, but this is used frequently within matplotlib: if np.prod(a.shape) == 0 : print "Is Empty!" Cheers! Ben Root
On Fri, Jan 27, 2012 at 20:46, Bartosz Telenczuk <b.telenczuk@biologie.hu-berlin.de> wrote:
I have been using numpy for several years and I am very impressed with its flexibility. However, there is one problem that has always bothered me.
Quite often I need to test consistently whether a variable is any of the following: an empty list, an empty array or None. Since both arrays and lists are ordered sequences I usually allow for both, and convert if necessary. However, when the (optional) argument is an empty list/array or None, I skip its processing and do nothing.
Now, how should I test for 'emptiness'?
PEP8 recommends:
For sequences, (strings, lists, tuples), use the fact that empty sequences are false.
seq = [] if not seq: ... print 'Hello'
It works for empty numpy arrays:
a = np.array(seq) if not a: ... print 'Hello" Hello
but if 'a' is non-empty it raises an exception:
a = np.array([1,2]) if not a: ... print 'Hello" ValueError: The truth value of an array with more than one element is ambiguous. Use a.any() or a.all()
One solution is to test lengths:
if len(seq) > 0: .... ... if len(a) > 0: ... ...
but for None it fails again:
opt = None if len(opt): ... TypeError: object of type 'NoneType' has no len()
even worse we can not test for None, because it will fail if someone accidentally wraps None in an array:
a = np.array(opt) if opt is not None: ... print 'hello' hello
Although this behaviour is expected, it may be very confusing and it easily leads to errors. Even worse it adds unnecessary complexity in the code, because arrays, lists and None have to be handled differently.
I hoped the I managed to explain the problem well. Is there a recommended way to test for empty arrays?
[~] |5> x = np.zeros([0]) [~] |6> x array([], dtype=float64) [~] |7> x.size == 0 True Note that checking for len(x) will fail for some empty arrays: [~] |8> x = np.zeros([10, 0]) [~] |9> x.size == 0 True [~] |10> len(x) 10 There is no way to test all of the cases (empty sequence, empty array, None) in the same way. Usually, it's a bad idea to conflate the three. -- Robert Kern "I have come to believe that the whole world is an enigma, a harmless enigma that is made terrible by our own mad attempt to interpret it as though it had an underlying truth." -- Umberto Eco
Thank you for your tips. I was not aware of the possible problems with len.
There is no way to test all of the cases (empty sequence, empty array, None) in the same way. Usually, it's a bad idea to conflate the three.
I agree that this should be avoided. However, there are cases in which it is not possible or hard. My case is that I get some extra data to add to my plots from a database. The dataset may be undefined (which means None), empty array or empty list. In all cases the data should not be plotted. If I want to test for all the cases, my program becomes quite complex. In fact, Python provides False values for most empty objects, but NumPy seems to ignore this. It might be a good idea to have a helper function which handles all objects consistently. Yours, Bartosz
On Fri, Jan 27, 2012 at 21:24, Bartosz Telenczuk <b.telenczuk@biologie.hu-berlin.de> wrote:
Thank you for your tips. I was not aware of the possible problems with len.
There is no way to test all of the cases (empty sequence, empty array, None) in the same way. Usually, it's a bad idea to conflate the three.
I agree that this should be avoided. However, there are cases in which it is not possible or hard. My case is that I get some extra data to add to my plots from a database. The dataset may be undefined (which means None), empty array or empty list. In all cases the data should not be plotted. If I want to test for all the cases, my program becomes quite complex.
Well, if you really need to do this in more than one place, define a utility function and call it a day. def should_not_plot(x): if x is None: return True elif isinstance(x, np.ndarray): return x.size == 0 else: return bool(x)
In fact, Python provides False values for most empty objects, but NumPy seems to ignore this. It might be a good idea to have a helper function which handles all objects consistently.
np.asarray(x).size == 0 None should rarely be treated the same as an empty list or a 0-size array, so that should be left to application-specific code. -- Robert Kern "I have come to believe that the whole world is an enigma, a harmless enigma that is made terrible by our own mad attempt to interpret it as though it had an underlying truth." -- Umberto Eco
On Fri, Jan 27, 2012 at 1:29 PM, Robert Kern <robert.kern@gmail.com> wrote:
Well, if you really need to do this in more than one place, define a utility function and call it a day.
def should_not_plot(x): if x is None: return True elif isinstance(x, np.ndarray): return x.size == 0 else: return bool(x)
I tend to do things like: def convert_to_plotable(x): if x is None: return None else: x = np.asarray(x) if b.size == 0: return None return x it does mean you need to check for None later anyway, but I like to convert to an array early in the process -- then you know you have either an array or None at that point. NOTE: you could also raise and handle an exception instead. -Chris -- Christopher Barker, Ph.D. Oceanographer Emergency Response Division NOAA/NOS/OR&R (206) 526-6959 voice 7600 Sand Point Way NE (206) 526-6329 fax Seattle, WA 98115 (206) 526-6317 main reception Chris.Barker@noaa.gov
participants (4)
-
Bartosz Telenczuk -
Benjamin Root -
Chris Barker -
Robert Kern