2011/10/18 Bruce Southey <bsouthey@gmail.com>

On 10/18/2011 09:12 AM, Chao YUE wrote:
thanks. Olivier. I see.

Chao

2011/10/18 Olivier Delalleau <shish@keba.be>

As far as I can tell ma.mean() is working as expected here: it computes the mean only over non-masked values.
If you want to get rid of any mean that was computed over a series containing masked value you can do:

b = a.mean(0)
b.mask[a.mask.any(0)] = True

Then b will be:

masked_array(data = [5.0 -- -- 8.0 9.0 -- 11.0 12.0 -- 14.0],
             mask = [False True True False False True False False True False],
       fill_value = 1e+20)

-=- Olivier

2011/10/18 Chao YUE <chaoyuejoy@gmail.com>

Dear all,

previoulsy I think np.ma.mean() will automatically filter the masked (missing) value but it's not?
In [489]: a=np.arange(20.).reshape(2,10)

In [490]: a=np.ma.masked_array(a,(a==2)|(a==5)|(a==11)|(a==18),fill_value=np.nan)

In [491]: a
Out[491]:
masked_array(data =
[[0.0 1.0 -- 3.0 4.0 -- 6.0 7.0 8.0 9.0]
[10.0 -- 12.0 13.0 14.0 15.0 16.0 17.0 -- 19.0]],
             mask =
[[False False True False False True False False False False]
[False True False False False False False False True False]],
       fill_value = nan)

In [492]: a.mean(0)
Out[492]:
masked_array(data = [5.0 1.0 12.0 8.0 9.0 15.0 11.0 12.0 8.0 14.0],
             mask = [False False False False False False False False False False],
       fill_value = 1e+20)

In [494]: np.ma.mean(a,0)
Out[494]:
masked_array(data = [5.0 1.0 12.0 8.0 9.0 15.0 11.0 12.0 8.0 14.0],
             mask = [False False False False False False False False False False],
       fill_value = 1e+20)

In [495]: np.ma.mean(a,0)==a.mean(0)
Out[495]:
masked_array(data = [ True True True True True True True True True True],
             mask = False,
       fill_value = True)

only use a.filled().mean(0) can I get the result I want:
In [496]: a.filled().mean(0)
Out[496]: array([ 5., NaN, NaN,   8.,   9., NaN, 11., 12., NaN, 14.])

I am doing this because I tried to have a small fuction from the web to do moving average for data:

import numpy as np
def rolling_window(a, window):
    if window < 1:
        raise ValueError, "`window` must be at least 1."
    if window > a.shape[-1]:
        raise ValueError, "`window` is too long."
    shape = a.shape[:-1] + (a.shape[-1] - window + 1, window)
    strides = a.strides + (a.strides[-1],)
    return np.lib.stride_tricks.as_strided(a, shape=shape, strides=strides)

def move_ave(a,window):
    temp=rolling_window(a,window)
    pre=int(window)/2
    post=int(window)-pre-1
    return np.concatenate((a[...,0:pre],np.mean(temp,-1),a[...,-post:]),axis=-1)

In [489]: a=np.arange(20.).reshape(2,10)

In [499]: move_ave(a,4)
Out[499]:
masked_array(data =
[[ 0.    1.    1.5   2.5   3.5   4.5   5.5   6.5   7.5   9. ]
[ 10.   11.   11.5 12.5 13.5 14.5 15.5 16.5 17.5 19. ]],
             mask =
False,
       fill_value = 1e+20)

thanks,

Chao

--

***********************************************************************************

Chao YUE
Laboratoire des Sciences du Climat et de l'Environnement (LSCE-IPSL)
UMR 1572 CEA-CNRS-UVSQ
Batiment 712 - Pe 119
91191 GIF Sur YVETTE Cedex

Tel: (33) 01 69 08 29 02; Fax:01.69.08.77.16

************************************************************************************

_______________________________________________
NumPy-Discussion mailing list
NumPy-Discussion@scipy.org
http://mail.scipy.org/mailman/listinfo/numpy-discussion

_______________________________________________
NumPy-Discussion mailing list
NumPy-Discussion@scipy.org
http://mail.scipy.org/mailman/listinfo/numpy-discussion

--

***********************************************************************************

Chao YUE
Laboratoire des Sciences du Climat et de l'Environnement (LSCE-IPSL)
UMR 1572 CEA-CNRS-UVSQ
Batiment 712 - Pe 119
91191 GIF Sur YVETTE Cedex

Tel: (33) 01 69 08 29 02; Fax:01.69.08.77.16

************************************************************************************
_______________________________________________
NumPy-Discussion mailing list
NumPy-Discussion@scipy.org
http://mail.scipy.org/mailman/listinfo/numpy-discussion
Looked at pandas for your rolling window functionality:
http://pandas.sourceforge.net
"Time series-specific functionality: date range generation and frequency conversion, moving window statistics, moving window linear regressions, date shifting and lagging, etc."

Bruce
_______________________________________________
NumPy-Discussion mailing list
NumPy-Discussion@scipy.org
http://mail.scipy.org/mailman/listinfo/numpy-discussion