[Numpy-discussion] Re: Using Reduce with Multi-dimensional Masked array

[Paul F. Dubois]

You have misread my reply. It is not true that MA.op works one way and
MA.op.reduce is different. sum and add.reduce are different, and the
documentation for sum 
DOES say the right thing for sum. The function sum is a special case in
that its native meaning was the same as add.reduce and so the function
is redundant.

I believe you are in error wrt average; average works the way you want. 
Function count can tell you the number of 
non-masked values either in the whole array or axis-wise if you give an
axis argument. Function size gives you the total number, so #invalid is
size(x)-count(x).

maximum and minimum (don't use max and min, they are built-ins that
don't know about Numeric)
have two forms. When called with one argument they return the overall
max or min of the whole array, returning masked only if all entries are
masked. For two arguments, you get element-wise extrema, and the mask is
on where any one of the arguments was masked.

>>> print x
[[1 ,-- ,3 ,]
 [11 ,-- ,-- ,]]
>>> print average(x)
[6.0 ,-- ,3.0 ,] 
>>> y
array(
 [[ 6, 7, 8,]
 [ 9,10,11,]])
>>> print maximum(x,y)
[[6 ,-- ,8 ,]
 [11 ,-- ,-- ,]]
>>> y[0,0]=masked
>>> print maximum(x,y)
[[-- ,-- ,8 ,]
 [11 ,-- ,-- ,]]
-----Original Message-----
From: numpy-discussion-admin at lists.sourceforge.net
[mailto:numpy-discussion-admin at lists.sourceforge.net] On Behalf Of Sue
Giller
Sent: Thursday, November 29, 2001 9:50 AM
To: numpy-discussion at lists.sourceforge.net
Subject: [Numpy-discussion] Re: Using Reduce with Multi-dimensional
Masked array


Thanks for the pointer.

The example I gave using the sum operation is merely an example - 
I could also be doing other manipulations such as min, max, 
average, etc.

I see that the MA.<op>.reduce functions will do what I want, but to 
do an average, I will need to do two steps since the MA.average 
function will have the original 'unexpected' behavior that I don't want.

That raises the question of how to determine a count of valid values 
in a masked array.  Can I assume that I can do 'math' on the mask 
array itself, for example to sum along a given axis and have the 
masked cells add up?

In my original example, I would expect a sum along the second axis 
to return [0,0,0,2,0].  Can I rely on this?  I would suggest that a 
.count operator would be very useful in working with masked arrays 
(count valid and count masked).

>>> m = MA.masked_values(a, -99)
>>> m
    array(data =
             [[  1,  2,  3,-99,  5,]
              [ 10, 20, 30,-99, 50,]],
           mask =
              [[0,0,0,1,0,]
               [0,0,0,1,0,]],
           fill_value=-99)

To add an opinion on the question from Paul about 'expected' 
behavior, I was working off the documentation for Numerical Python, 
and there were no caveats in there about MA.<op> working one 
way, and MA.<op>.reduce working another.  The answer is always 
in the documentation, especially for users like me who don't have 
time or knkowledge to go reading thru all the code modules to try 
and figure out what is happening.  From a purely user standpoint, I 
would expect a masked array to retain it's mask-edness at all times, 
unless I explicitly tell it not to.  In that case, I would still expect
it to 
replace the 'masked' cells with the original masked value, and not 
just arbitrarily assign some other value, such as 0.

Thanks again for the prompt reply.


_______________________________________________
Numpy-discussion mailing list Numpy-discussion at lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/numpy-discussion