change masked array member values with conditional selection
Dear all numpy users, I am using numpy 1.6.1 I find that if you want to change some member values in a masked array according to some conditional selection. suppose a is a masked array, you want to change all value below zero to zero. you must always use a[np.nonzero(a<0)]=0 rather than a[a<0]=0. the latter will lose all masked elements. an example: In [24]: a=np.arange(10.) In [25]: a=np.ma.masked_array(a,mask=a<3) In [28]: a[a<5]=2. In [29]: a Out[29]: masked_array(data = [2.0 2.0 2.0 2.0 2.0 5.0 6.0 7.0 8.0 9.0], mask = [False False False False False False False False False False], fill_value = 1e+20) In [30]: b=np.arange(10.) In [31]: b=np.ma.masked_array(b,mask=b<3) In [34]: b[np.nonzero(b<5)]=2. In [35]: b Out[35]: masked_array(data = [-- -- -- 2.0 2.0 5.0 6.0 7.0 8.0 9.0], mask = [ True True True False False False False False False False], fill_value = 1e+20) cheers, Chao -- *********************************************************************************** Chao YUE Laboratoire des Sciences du Climat et de l'Environnement (LSCE-IPSL) UMR 1572 CEA-CNRS-UVSQ Batiment 712 - Pe 119 91191 GIF Sur YVETTE Cedex Tel: (33) 01 69 08 29 02; Fax:01.69.08.77.16 ************************************************************************************
Ciao Chao, That known quirk deserves to be better documented, I agree. There's a simple explanation for this behavior: Because `a` is a masked array, `(a < 5)` is also a masked array with dtype=np.bool, and whose mask is the same as `a`'s. In your example, that's: masked_array(data = [-- -- -- True True False False False False False], mask = [ True True True False False False False False False False], fill_value = True) Now, what should we do with the masked entries ? Should we consider them as False? As True? That's up to you, actually... Because it's never a good idea to use masked arrays as condition (as you just experienced), I advise you to be explicit. In your case, that'd be
a[(a<5).filled(False)] = 2
If you go in the source code of numpy.ma.core, in the __getitem__/__setitem__ methods, you'll find a little warning that I commented (because numpy.ma is already slow enough that I didn't want to make it even slower)... On 4/12/12, Chao YUE <chaoyuejoy@gmail.com> wrote:
Dear all numpy users,
I am using numpy 1.6.1
I find that if you want to change some member values in a masked array according to some conditional selection. suppose a is a masked array, you want to change all value below zero to zero. you must always use
a[np.nonzero(a<0)]=0
rather than a[a<0]=0.
the latter will lose all masked elements.
an example: In [24]: a=np.arange(10.)
In [25]: a=np.ma.masked_array(a,mask=a<3)
In [28]: a[a<5]=2.
In [29]: a Out[29]: masked_array(data = [2.0 2.0 2.0 2.0 2.0 5.0 6.0 7.0 8.0 9.0], mask = [False False False False False False False False False False], fill_value = 1e+20)
In [30]: b=np.arange(10.)
In [31]: b=np.ma.masked_array(b,mask=b<3)
In [34]: b[np.nonzero(b<5)]=2.
In [35]: b Out[35]: masked_array(data = [-- -- -- 2.0 2.0 5.0 6.0 7.0 8.0 9.0], mask = [ True True True False False False False False False False], fill_value = 1e+20)
cheers,
Chao -- *********************************************************************************** Chao YUE Laboratoire des Sciences du Climat et de l'Environnement (LSCE-IPSL) UMR 1572 CEA-CNRS-UVSQ Batiment 712 - Pe 119 91191 GIF Sur YVETTE Cedex Tel: (33) 01 69 08 29 02; Fax:01.69.08.77.16 ************************************************************************************
Thanks Pierre. It's a good idea to always use a[(a<5).filled(False)] = 2 I don't understand very well the underlying structure but it's good to know some. Chao 2012/4/12 Pierre GM <pgmdevlist@gmail.com>
Ciao Chao,
That known quirk deserves to be better documented, I agree.
There's a simple explanation for this behavior: Because `a` is a masked array, `(a < 5)` is also a masked array with dtype=np.bool, and whose mask is the same as `a`'s. In your example, that's: masked_array(data = [-- -- -- True True False False False False False], mask = [ True True True False False False False False False False], fill_value = True) Now, what should we do with the masked entries ? Should we consider them as False? As True? That's up to you, actually... Because it's never a good idea to use masked arrays as condition (as you just experienced), I advise you to be explicit. In your case, that'd be
a[(a<5).filled(False)] = 2
If you go in the source code of numpy.ma.core, in the __getitem__/__setitem__ methods, you'll find a little warning that I commented (because numpy.ma is already slow enough that I didn't want to make it even slower)...
On 4/12/12, Chao YUE <chaoyuejoy@gmail.com> wrote:
Dear all numpy users,
I am using numpy 1.6.1
I find that if you want to change some member values in a masked array according to some conditional selection. suppose a is a masked array, you want to change all value below zero to zero. you must always use
a[np.nonzero(a<0)]=0
rather than a[a<0]=0.
the latter will lose all masked elements.
an example: In [24]: a=np.arange(10.)
In [25]: a=np.ma.masked_array(a,mask=a<3)
In [28]: a[a<5]=2.
In [29]: a Out[29]: masked_array(data = [2.0 2.0 2.0 2.0 2.0 5.0 6.0 7.0 8.0 9.0], mask = [False False False False False False False False False False], fill_value = 1e+20)
In [30]: b=np.arange(10.)
In [31]: b=np.ma.masked_array(b,mask=b<3)
In [34]: b[np.nonzero(b<5)]=2.
In [35]: b Out[35]: masked_array(data = [-- -- -- 2.0 2.0 5.0 6.0 7.0 8.0 9.0], mask = [ True True True False False False False False False False], fill_value = 1e+20)
cheers,
Chao --
***********************************************************************************
Chao YUE Laboratoire des Sciences du Climat et de l'Environnement (LSCE-IPSL) UMR 1572 CEA-CNRS-UVSQ Batiment 712 - Pe 119 91191 GIF Sur YVETTE Cedex Tel: (33) 01 69 08 29 02; Fax:01.69.08.77.16
************************************************************************************
_______________________________________________ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
-- *********************************************************************************** Chao YUE Laboratoire des Sciences du Climat et de l'Environnement (LSCE-IPSL) UMR 1572 CEA-CNRS-UVSQ Batiment 712 - Pe 119 91191 GIF Sur YVETTE Cedex Tel: (33) 01 69 08 29 02; Fax:01.69.08.77.16 ************************************************************************************
participants (2)
-
Chao YUE -
Pierre GM