Folks, You can find a new patch for MA on the wiki http://projects.scipy.org/scipy/numpy/attachment/wiki/MaskedArray/ma-2006032... along with a test suite. The 'clip' method should now work with array arguments. Were also added cumsum, cumprod, std, var and squeeze. I'll deal with flags, setflags, setfield, dump and others when I'll have a better idea of how it works -- which probably won't happen anytime soon, as I don't really have time to dig in the code for these functions. AAMOF, I'm more interested in checking/patching some other aspects of numpy for MA (eg, mlab...) Once again, please send me your comments and suggestions. Thx for everything P.
I have applied the patch with minor modifications. See < http://projects.scipy.org/scipy/numpy/changeset/2331>. Here are a few suggestions for posting patches. 1. If you are using svn, please post output of "svn diff" in the project root directory (the directory that *contains* "numpy", not the "numpy" directory. 2. If appropriate, add unit tests to an existing file instead of creating a new one. (In case of ma, the correct file is test_ma.py). 3. If you follow recommendation #1, this will happen automatically, if you cannot use svn for some reason, concatenate the output of diff for code and test in the same patch file. Here are some topics for discussion. 1. I've initially implemented some ma array methods by wrapping existing module level functions. I am not sure this is the best approach to implement new methods. It is probably cleaner to implement them as methods and provide wrappers at the module level similar to oldnumeric. 2. I am not sure cumprod and cumsum should fill masked elements with 1 and 0. I would think the result should be masked if any prior element along the axis being accumulated is masked. To ignore masked elements, filled can be called explicitly before cum[prod|sum]. One of the problems with filling by default is that 1 or 0 are not appropriate values for object arrays (for example, "" is an appropriate fill value for cumsum of an array of strings). On 3/28/06, Pierre GM <pgmdevlist@mailcan.com> wrote:
Folks, You can find a new patch for MA on the wiki
http://projects.scipy.org/scipy/numpy/attachment/wiki/MaskedArray/ma-2006032... along with a test suite. The 'clip' method should now work with array arguments. Were also added cumsum, cumprod, std, var and squeeze. I'll deal with flags, setflags, setfield, dump and others when I'll have a better idea of how it works -- which probably won't happen anytime soon, as I don't really have time to dig in the code for these functions. AAMOF, I'm more interested in checking/patching some other aspects of numpy for MA (eg, mlab...) Once again, please send me your comments and suggestions. Thx for everything P.
------------------------------------------------------- This SF.Net email is sponsored by xPML, a groundbreaking scripting language that extends applications into web and mobile media. Attend the live webcast and join the prime developer group breaking into this new coding territory! http://sel.as-us.falkag.net/sel?cmd=lnk&kid=110944&bid=241720&dat=121642 _______________________________________________ Numpy-discussion mailing list Numpy-discussion@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/numpy-discussion
On 4/7/06, Sasha <ndarray@mac.com> wrote:
2. I am not sure cumprod and cumsum should fill masked elements with 1 and 0. I would think the result should be masked if any prior element along the axis being accumulated is masked. To ignore masked elements, filled can be called explicitly before cum[prod|sum]. One of the problems with filling by default is that 1 or 0 are not appropriate values for object arrays (for example, "" is an appropriate fill value for cumsum of an array of strings).
There are often a number of options for how masked values can be dealt with. In general (not just with cum*), I would prefer for the result to be masked when masked values are involved unless I explicitly indicate what should be done with the masked values. Otherwise it is too easy to forget that some default maniputlation of masked values has been applied. In R there is commonly an na.action or na.rm parameter to functions. Michael
On 4/6/06, Michael Sorich <michael.sorich@gmail.com> wrote:
... I would prefer for the result to be masked when masked values are involved unless I explicitly indicate what should be done with the masked values. ...
This is the case in r2332:
from numpy.core.ma import * print array([1,2,3], mask=[0,1,0]).cumsum() [1 -- --]
Sasha, Thanks for your advice with SVN. I'll make sure to use that method from now on.
1. I've initially implemented some ma array methods by wrapping existing module level functions. I am not sure this is the best approach to implement new methods. It is probably cleaner to implement them as methods and provide wrappers at the module level similar to oldnumeric.
Well, I tried to stick to the latest convention, getting rid of the _wrapit part. Let me know.
2. I am not sure cumprod and cumsum should fill masked elements with 1 and 0.
Good point for the object/string arrays, yet other cases I overlooked (I'm still not used to object arrays, I'm now realizing they're quite useful). Actually, I coded that way because it's how I use these functions. But well, as many settings as users, eh? Michael's suggestion of introducing R-like options sounds interesting, but I wonder whether it would not be a bit heavy for methods, with the introduction of an extra flag. That'd be great for functions, though. So, for cumsum and cumprod methods, maybe we could stick to Sasha's and Michael's preference (mask all values after the first missing), and we would just have to create two functions. We could use the 4 R ones: na.omit, na.fail, na.pass, na.exclude. For our current problem (cumsum,cumprod) na.omit: would return the result I implemented (fill with 0 or 1) na.fail: would return masked values after the first missing na.exclude: would correspond to compressed().cumsum() ? I don't like that, it changes the initial length/size na.pass: I don't know...
participants (3)
-
Michael Sorich -
Pierre GM -
Sasha