[Numpy-discussion] Why does fancy indexing work like this?

[Sebastian Berg]

On Wed, 2020-07-22 at 17:35 -0600, Aaron Meurer wrote:
> > About your warnings, do you have a nice way to do that?  The
> > mechanism
> > for warnings does not really give a good way to catch that a
> > warning
> > was raised and then turn it into an error.  Unless someone
> > contributes
> > a slick way to do it, I am not sure the complexity pays off.
> 
> I don't really know how flags and options and such work in NumPy, but
> I would imagine something like
> 
> if flags['post-deprecation'] = True: # Either a single flag for all
> deprecations or a per-deprecation flag
>     raise IndexError(...)
> else:
>     warnings.warn(...)
> 

We have never done global flags for these things much in NumPy, I don't
know of precedence in other packages, possibly aside future imports,
but I am not even sure they have been used in this way.

> I don't know if the fact that the code that does this is in C
> complicates things.
> 
> In other words, something that works kind of like __future__ flags
> for
> upgrading the behavior to post-deprecation.
> 
> > IIRC, I added the note about raising the warning, because in this
> > particular case the deprecation warning (turned into an error)
> > happens
> > to be chained due to implementation details.  (so you do see the
> > "original" error printed out).
> 
> Yes, it's nice that you can see it. But for my use case, I want to be
> able to "except IndexError". Basically, for ndindex, I test against
> NumPy to make sure the semantics are identical, and that includes
> making sure identical exceptions are raised. I also want to make it
> so
> that the ndindex semantics always follow post-deprecation behavior
> for
> any NumPy deprecations, since that leads to a cleaner API. But that
> means that my test code has to do fancy shenanigans to catch these
> deprecation warnings and treat them like the right errors.
> 
> But even as a general principle, I think for any deprecation warning,
> users should be able to update their code in such a way that the
> current version doesn't give the warning and also it will continue to

For FutureWarnings, I will always try very hard to give an option to
opt-in to new behaviour or old behaviour – ideally with code compatible
also with earlier NumPy versions.

Here, for a DeprecationWarning that has obviously no "alternative", I
cannot think of any precedence in any other package or Python itself
doing such a dance.  And it is extremely fringe (you only need it
because you are testing another package against numpy behaviour!).

So I am happy to merge it if its proposed (maybe its easier for you to
add this to NumPy then work around it in your tests), but I am honestly
concerned that proposing this as a general principle is far more churn
then worth the trouble.  At least unless there is some consensus (and
probably precendence in the scientific python ecosystem or python
itself).

Cheers,

Sebastian


> work and be idiomatic for future versions. For simple deprecations
> where you remove a function x(), this is often as simple as telling
> people to replace x() with y(). But these deprecations aren't so
> simple, because the indexing itself is valid and will stay valid,
> it's
> just the behavior that will change. If there's no way to do this,
> then
> a deprecation warning serves little purpose because users who see the
> warning won't be able to do anything about it until things actually
> change. There would be little difference from just changing things
> outright. For the list as tuple indexing thing, you can already kind
> of do this by making sure your fancy indices are always arrays. For
> the out of bounds one, it's a little harder. I guess for most
> use-cases, you aren't actually checking for IndexErrors, and the
> thing
> that will become an error usually indicates a bug in user code, so
> maybe it isn't a huge deal (I admit my use-cases aren't typical).
> 
> Aaron Meurer
> _______________________________________________
> NumPy-Discussion mailing list
> NumPy-Discussion at python.org
> https://mail.python.org/mailman/listinfo/numpy-discussion
> 

-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 833 bytes
Desc: This is a digitally signed message part
URL: <http://mail.python.org/pipermail/numpy-discussion/attachments/20200723/e565a106/attachment.sig>