Non-meta indexing improvements discussion
Hi all,
Let me take a shot at summing up some suggestions to make the indexing less surprising, and maybe we can gather some more in a more concentrated way now.
1. Implement something like `arr.fancy_index[...]` and `arr.ortho_index[...]` (i.e. Jaimes PR is the start for trying this)
2. Add warnings for non-consecutive advanced indexing (i.e. the original example `arr[0, :, index_array]`).
3. I do not know if it possible or useful, but I could imagine a module wide switch (similar to __future__ imports) to change the default indexing behaviour.
One more thing, implementing this (especially the "new" indexing) is non-trivial, so as always help beyond just a discussion is appreciated and in my opinion the best way to push an actual change to happen sooner rather then in some far off future. I do not have time for concentrating much on an implementation for a while myself for a while at least.
- Sebastian
On Do, 2015-04-09 at 08:50 +0200, Sebastian Berg wrote:
Hi all,
Let me take a shot at summing up some suggestions to make the indexing less surprising, and maybe we can gather some more in a more concentrated way now.
Did not want to comment on the first mail
- Implement something like `arr.fancy_index[...]` and
`arr.ortho_index[...]` (i.e. Jaimes PR is the start for trying this)
I like this, personally. There is not much to be lost and I fully agree with Robert on this. It opens up a lot of possibilities for us and especially also others.
- Add warnings for non-consecutive advanced indexing (i.e. the original
example `arr[0, :, index_array]`).
This could be annoying sometimes, but then warnings do not break legacy code, and I think in new code again Robert is right for these cases using arr.fancy_index[...] is more explicit and a nice warning/google help to the confused reader.
- I do not know if it possible or useful, but I could imagine a module
wide switch (similar to __future__ imports) to change the default indexing behaviour.
OK, my suggestion.... But actually I do not know if I like it all that much (nor if it can be done) since 1. and 2. seem to me like enough. But if someone feels strongly about fancy indexing being bad, I wanted to point out that there may be ways to go down the road to "switch" numpy without actually switching.
- Sebastian
One more thing, implementing this (especially the "new" indexing) is non-trivial, so as always help beyond just a discussion is appreciated and in my opinion the best way to push an actual change to happen sooner rather then in some far off future. I do not have time for concentrating much on an implementation for a while myself for a while at least.
- Sebastian
NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
On Thu, Apr 9, 2015 at 8:07 AM, Sebastian Berg sebastian@sipsolutions.net wrote:
On Do, 2015-04-09 at 08:50 +0200, Sebastian Berg wrote:
- I do not know if it possible or useful, but I could imagine a module
wide switch (similar to __future__ imports) to change the default indexing behaviour.
OK, my suggestion.... But actually I do not know if I like it all that much (nor if it can be done) since 1. and 2. seem to me like enough. But if someone feels strongly about fancy indexing being bad, I wanted to point out that there may be ways to go down the road to "switch" numpy without actually switching.
I can't think of a way to actually make that work (I can list all the ways I thought of that *don't* work if anyone insists, but it's a tedious dead end), but it also seems to me to be a step backward. Assuming that we have both .ortho_ix and .fancy_ix to work with, it seems to me that the explicitness is a good thing. Even if in this module you only want to exploit one of those semantics, your module's readers live in a wider context where both semantics play a role. Moving your marker of "this is the non-default semantics I am using" to some module or function header away from where the semantics are actually used makes the code harder to read.
A newish user trying to read some nontrivial indexing code may come to the list and ask "what exactly is this expression doing here?" and give us just the line of code with the expression (anecdotally, this is usually how this scenario goes down). If we have to answer "it depends; is there an @ortho_indexing decorator at the top of the function?", that's probably a cure worse than the disease. The properties are a good way to provide googleable signposts right where the tricky semantics are being used.
-- Robert Kern
On Wed, Apr 8, 2015 at 11:50 PM, Sebastian Berg sebastian@sipsolutions.net wrote:
Hi all,
Let me take a shot at summing up some suggestions to make the indexing less surprising, and maybe we can gather some more in a more concentrated way now.
- Implement something like `arr.fancy_index[...]` and
`arr.ortho_index[...]` (i.e. Jaimes PR is the start for trying this)
- Add warnings for non-consecutive advanced indexing (i.e. the original
example `arr[0, :, index_array]`).
- I do not know if it possible or useful, but I could imagine a module
wide switch (similar to __future__ imports) to change the default indexing behaviour.
One more thing, implementing this (especially the "new" indexing) is non-trivial, so as always help beyond just a discussion is appreciated and in my opinion the best way to push an actual change to happen sooner rather then in some far off future. I do not have time for concentrating much on an implementation for a while myself for a while at least.
I don't think that should be the biggest hurdle if we decided to go for it. I'm pretty sure that, with some adult supervision, I could pull that one off. And you don't always get a chance to put your hands on code as cool as numpy's indexing! ;-)
But before we go down that route, I think we should spend a non-trivial amount of effort on figuring out exactly what we want. While the proposal that has started to emerge from this discussion, with the two indexers, sort of makes sense, we may very well be doing this https://xkcd.com/927/ ...
To paraphrase General Ripper https://www.youtube.com/watch?v=N1KvgtEnABY&t=31s, I have neither the time, the training, nor the inclination for strategic thought. But we should probably write this into a NEP, and share it with the involved parties (netcdf, xay, pandas, Blaze, dynd...) and try to come to some agreement. We are already late for SciPy15, but perhaps a talk on the proposal would be in order for EuroSciPy, or for one of the PyData conferences, to try and gather some feedback from potential users.
Without that, this isn't going anywhere...
Jaime
participants (3)
-
Jaime Fernández del Río -
Robert Kern -
Sebastian Berg