On May 4, 2010, at 8:38 PM, Gökhan Sever wrote:

Hello,

I have the following arrays read as masked array.

I[10]: basic.data['Air_Temp'].mask O[10]: array([ True, False, False, ..., False, False, False], dtype=bool)

[12]: basic.data['Press_Alt'].mask O[12]: False

I[13]: len basic.data['Air_Temp'] -----> len(basic.data['Air_Temp']) O[13]: 1758

The first item data['Air_Temp'] has only the first element masked and this result with mask attribute being created an equal data length bool array. On the other hand data['Press_Alt'] has no elements to mask yielding a 'False' scalar. Is this a documented behavior or intentionally designed this way? This is the only case out of 20 that breaks my code as following: :)

IndexError Traceback (most recent call last)

130 for k in range(len(shorter)): 131 if (serialh.data['dccnTempSF'][k] != 0) \ --> 132 and (basic.data['Air_Temp'].mask[k+diff] == False): 133 dccnConAmb[k] = serialc.data['dccnConc'][k] * \ 134 physical.data['STATIC_PR'][k+diff] * \

IndexError: invalid index to scalar variable.

since mask is a scalar in this case, nothing to loop terminating with an IndexError.

Gokhan, Sorry for not getting back sooner, web connectivity was limited on my side. I must admit I can't really see what you're tring to do here, but I'll throw some random comments: * If you're using structured MaskedArrays, it's a really bad idea to call one of the fields "data", as it may interact in a non-obvious way with the actual "data" property (the one that outputs a view of the array as a pure ndarray). * if you need to test whether an array has some masked elements, try something like

...

...

myarray.mask is nomask If True, no item is masked, the mask is a boolean and you can move on. If False, then the mask is a ndarray w/ as many elements as the array and you can index it.

On 05/09/2010 09:01 AM, Gökhan Sever wrote:

On Fri, May 7, 2010 at 3:28 PM, Pierre GM mailto:pgmdevlist@gmail.com> wrote:

On May 4, 2010, at 8:38 PM, Gökhan Sever wrote: > Hello, > > I have the following arrays read as masked array. > > I[10]: basic.data['Air_Temp'].mask > O[10]: array([ True, False, False, ..., False, False, False], dtype=bool) > > [12]: basic.data['Press_Alt'].mask > O[12]: False > > I[13]: len basic.data['Air_Temp'] > -----> len(basic.data['Air_Temp']) > O[13]: 1758 > > > The first item data['Air_Temp'] has only the first element masked and this result with mask attribute being created an equal data length bool array. On the other hand data['Press_Alt'] has no elements to mask yielding a 'False' scalar. Is this a documented behavior or intentionally designed this way? This is the only case out of 20 that breaks my code as following: :) > > IndexError Traceback (most recent call last) > > 130 for k in range(len(shorter)): > 131 if (serialh.data['dccnTempSF'][k] != 0) \ > --> 132 and (basic.data['Air_Temp'].mask[k+diff] == False): > 133 dccnConAmb[k] = serialc.data['dccnConc'][k] * \ > 134 physical.data['STATIC_PR'][k+diff] * \ > > IndexError: invalid index to scalar variable. > > since mask is a scalar in this case, nothing to loop terminating with an IndexError.

Gokhan, Sorry for not getting back sooner, web connectivity was limited on my side. I must admit I can't really see what you're tring to do here, but I'll throw some random comments: * If you're using structured MaskedArrays, it's a really bad idea to call one of the fields "data", as it may interact in a non-obvious way with the actual "data" property (the one that outputs a view of the array as a pure ndarray).

Hello Pierre,

basic.data is a dictionary containing all masked array items. When I read the original data into scripts, my main constructor-reader class automatically converts data to masked arrays. basic.data['Air_Temp'] is a masked array itself, little confusing for sure it also has 'data' attribute.

In the above example I check one condition looping in mask value. When mask attribute isn't an bool-array (when there is no missing value in data) the condition fails asserting an IndexError. I was wondering why it doesn't yield a bool-array instead of giving me a scalar False.

The mask attribute can be a full array, or it can be a scalar to indicate that nothing is masked. This is an optimization in masked arrays; it adds complexity, but it can save space and/or processing time. You can always access a full mask array by using np.ma.getmaskarray(). Or you can ensure the internal mask is an array, not a scalar, by using the shrink=False kwarg when making the masked array with np.ma.array(). Offhand, I suspect your loop can be eliminated by vectorization. Something like this: ns = len(shorter) slice0 = slice(ns) slice1 = slice(diff, diff+ns) cond1 = serialh.data['dccnTempSF'][slice0] != 0 cond2 = np.ma.getmaskarray(basic.data['Air_Temp'][slice1]) == False cond = cond1 & cond2 dccnConAmb[slice0][cond] = (serialc.data['dccnConc'][slice0][cond] * physical.data['STATIC_PR'][slice1][cond]) Eric