Putting in `np.ma.ndenumerate` MaskedArray specific ndenumerate
Hi all, the `np.ndenumerate` does not work well for masked arrays (like many main namespace functions, it simply ignores/drops the mask). There is a PR (https://github.com/numpy/numpy/pull/20020) to add a version of it to `np.ma` (masked array specific). And we thought it seemed reasonable and were planning on putting it in. This version skips all masked elements. An alternative could be to return `np.ma.masked` for masked elements? So if anyone thinks that may be the better solution, please send a brief mail. (Personally, I don't have opinions on masked arrays for the most part.) Cheers, Sebastian
On Wed, Nov 17, 2021 at 7:39 PM Sebastian Berg <sebastian@sipsolutions.net> wrote:
Hi all,
the `np.ndenumerate` does not work well for masked arrays (like many main namespace functions, it simply ignores/drops the mask).
There is a PR (https://github.com/numpy/numpy/pull/20020) to add a version of it to `np.ma` (masked array specific). And we thought it seemed reasonable and were planning on putting it in.
This version skips all masked elements. An alternative could be to return `np.ma.masked` for masked elements?
So if anyone thinks that may be the better solution, please send a brief mail.
Would it be a bad idea to add a kwarg that specifies this behaviour (i.e. offering both alternatives)? Assuming people might need the masked items to be there under certain circumstances. Perhaps when zipping masked data with dense data? András
(Personally, I don't have opinions on masked arrays for the most part.)
Cheers,
Sebastian _______________________________________________ NumPy-Discussion mailing list -- numpy-discussion@python.org To unsubscribe send an email to numpy-discussion-leave@python.org https://mail.python.org/mailman3/lists/numpy-discussion.python.org/ Member address: deak.andris@gmail.com
On Wed, 2021-11-17 at 19:49 +0100, Andras Deak wrote:
On Wed, Nov 17, 2021 at 7:39 PM Sebastian Berg <sebastian@sipsolutions.net> wrote:
Hi all,
the `np.ndenumerate` does not work well for masked arrays (like many main namespace functions, it simply ignores/drops the mask).
There is a PR (https://github.com/numpy/numpy/pull/20020) to add a version of it to `np.ma` (masked array specific). And we thought it seemed reasonable and were planning on putting it in.
This version skips all masked elements. An alternative could be to return `np.ma.masked` for masked elements?
So if anyone thinks that may be the better solution, please send a brief mail.
Would it be a bad idea to add a kwarg that specifies this behaviour (i.e. offering both alternatives)? Assuming people might need the masked items to be there under certain circumstances. Perhaps when zipping masked data with dense data?
Sure, if you agree the default should be skipping, I guess we are OK with adding it? ;) Cheers, Sebastian
András
(Personally, I don't have opinions on masked arrays for the most part.)
Cheers,
Sebastian _______________________________________________ NumPy-Discussion mailing list -- numpy-discussion@python.org To unsubscribe send an email to numpy-discussion-leave@python.org https://mail.python.org/mailman3/lists/numpy-discussion.python.org/ Member address: deak.andris@gmail.com
_______________________________________________ NumPy-Discussion mailing list -- numpy-discussion@python.org To unsubscribe send an email to numpy-discussion-leave@python.org https://mail.python.org/mailman3/lists/numpy-discussion.python.org/ Member address: sebastian@sipsolutions.net
On Wed, Nov 17, 2021 at 8:35 PM Sebastian Berg <sebastian@sipsolutions.net> wrote:
On Wed, 2021-11-17 at 19:49 +0100, Andras Deak wrote:
On Wed, Nov 17, 2021 at 7:39 PM Sebastian Berg <sebastian@sipsolutions.net> wrote:
Hi all,
the `np.ndenumerate` does not work well for masked arrays (like many main namespace functions, it simply ignores/drops the mask).
There is a PR (https://github.com/numpy/numpy/pull/20020) to add a version of it to `np.ma` (masked array specific). And we thought it seemed reasonable and were planning on putting it in.
This version skips all masked elements. An alternative could be to return `np.ma.masked` for masked elements?
So if anyone thinks that may be the better solution, please send a brief mail.
Would it be a bad idea to add a kwarg that specifies this behaviour (i.e. offering both alternatives)? Assuming people might need the masked items to be there under certain circumstances. Perhaps when zipping masked data with dense data?
Sure, if you agree the default should be skipping, I guess we are OK with adding it? ;)
I don't actually use masked arrays myself, nor ndenumerate, so I'm very forgiving in this matter... But if both use cases are plausible (_if_, although I can indeed imagine that this is the case), supporting both seems straightforward. Considering the pure python implementation it wouldn't be a problem to expose both functionalities. András
Cheers,
Sebastian
András
(Personally, I don't have opinions on masked arrays for the most part.)
Cheers,
Sebastian _______________________________________________ NumPy-Discussion mailing list -- numpy-discussion@python.org To unsubscribe send an email to numpy-discussion-leave@python.org https://mail.python.org/mailman3/lists/numpy-discussion.python.org/ Member address: deak.andris@gmail.com
_______________________________________________ NumPy-Discussion mailing list -- numpy-discussion@python.org To unsubscribe send an email to numpy-discussion-leave@python.org https://mail.python.org/mailman3/lists/numpy-discussion.python.org/ Member address: sebastian@sipsolutions.net
_______________________________________________ NumPy-Discussion mailing list -- numpy-discussion@python.org To unsubscribe send an email to numpy-discussion-leave@python.org https://mail.python.org/mailman3/lists/numpy-discussion.python.org/ Member address: deak.andris@gmail.com
I think a separate ndenumerate() in the masked array namespace would make a lot of sense. This is much less risky than changing np.ndenumerate(). On Wed, Nov 17, 2021 at 11:54 AM Andras Deak <deak.andris@gmail.com> wrote:
On Wed, Nov 17, 2021 at 8:35 PM Sebastian Berg <sebastian@sipsolutions.net> wrote:
On Wed, 2021-11-17 at 19:49 +0100, Andras Deak wrote:
On Wed, Nov 17, 2021 at 7:39 PM Sebastian Berg <sebastian@sipsolutions.net> wrote:
Hi all,
the `np.ndenumerate` does not work well for masked arrays (like many main namespace functions, it simply ignores/drops the mask).
There is a PR (https://github.com/numpy/numpy/pull/20020) to add a version of it to `np.ma` (masked array specific). And we thought it seemed reasonable and were planning on putting it in.
This version skips all masked elements. An alternative could be to return `np.ma.masked` for masked elements?
So if anyone thinks that may be the better solution, please send a brief mail.
Would it be a bad idea to add a kwarg that specifies this behaviour (i.e. offering both alternatives)? Assuming people might need the masked items to be there under certain circumstances. Perhaps when zipping masked data with dense data?
Sure, if you agree the default should be skipping, I guess we are OK with adding it? ;)
I don't actually use masked arrays myself, nor ndenumerate, so I'm very forgiving in this matter... But if both use cases are plausible (_if_, although I can indeed imagine that this is the case), supporting both seems straightforward. Considering the pure python implementation it wouldn't be a problem to expose both functionalities.
András
Cheers,
Sebastian
András
(Personally, I don't have opinions on masked arrays for the most part.)
Cheers,
Sebastian _______________________________________________ NumPy-Discussion mailing list -- numpy-discussion@python.org To unsubscribe send an email to numpy-discussion-leave@python.org https://mail.python.org/mailman3/lists/numpy-discussion.python.org/ Member address: deak.andris@gmail.com
_______________________________________________ NumPy-Discussion mailing list -- numpy-discussion@python.org To unsubscribe send an email to numpy-discussion-leave@python.org https://mail.python.org/mailman3/lists/numpy-discussion.python.org/ Member address: sebastian@sipsolutions.net
_______________________________________________ NumPy-Discussion mailing list -- numpy-discussion@python.org To unsubscribe send an email to numpy-discussion-leave@python.org https://mail.python.org/mailman3/lists/numpy-discussion.python.org/ Member address: deak.andris@gmail.com
_______________________________________________ NumPy-Discussion mailing list -- numpy-discussion@python.org To unsubscribe send an email to numpy-discussion-leave@python.org https://mail.python.org/mailman3/lists/numpy-discussion.python.org/ Member address: shoyer@gmail.com
On Wed, Nov 17, 2021 at 7:51 PM Andras Deak <deak.andris@gmail.com> wrote:
On Wed, Nov 17, 2021 at 7:39 PM Sebastian Berg <sebastian@sipsolutions.net> wrote:
Hi all,
the `np.ndenumerate` does not work well for masked arrays (like many main namespace functions, it simply ignores/drops the mask).
There is a PR (https://github.com/numpy/numpy/pull/20020) to add a version of it to `np.ma` (masked array specific). And we thought it seemed reasonable and were planning on putting it in.
This version skips all masked elements. An alternative could be to return `np.ma.masked` for masked elements?
Would it be a bad idea to add a kwarg that specifies this behaviour (i.e. offering both alternatives)? Assuming people might need the masked items to be there under certain circumstances. Perhaps when zipping masked data with dense data?
Just one remark about this. I chose not to yield masked elements, because it would basically turn the implementation into for index in np.ndindex(a.shape): yield index, a[index] Rather than zipping, you could as well iterate over `np.ndindex` and get the respective items from the dense and masked arrays via indexing. Really only when skipping masked elements does `ndenumerate` offer some meaningful MaskedArray-specific functionality. I'm not saying the alternative behavior should not be supported (although I personally see little benefit), but the above argument should at least be convincing that skipping is a useful default. Regards, - Jouke
On Thu, Nov 18, 2021 at 1:46 PM Jouke Witteveen <j.witteveen@gmail.com> wrote:
On Wed, Nov 17, 2021 at 7:51 PM Andras Deak <deak.andris@gmail.com> wrote:
On Wed, Nov 17, 2021 at 7:39 PM Sebastian Berg <sebastian@sipsolutions.net> wrote:
Hi all,
the `np.ndenumerate` does not work well for masked arrays (like many main namespace functions, it simply ignores/drops the mask).
There is a PR (https://github.com/numpy/numpy/pull/20020) to add a version of it to `np.ma` (masked array specific). And we thought it seemed reasonable and were planning on putting it in.
This version skips all masked elements. An alternative could be to return `np.ma.masked` for masked elements?
Would it be a bad idea to add a kwarg that specifies this behaviour (i.e. offering both alternatives)? Assuming people might need the masked items to be there under certain circumstances. Perhaps when zipping masked data with dense data?
I appended a commit to the PR that implements the kwarg for optionally yielding masked elements. Regards, - Jouke
participants (4)
-
Andras Deak -
Jouke Witteveen -
Sebastian Berg -
Stephan Hoyer