[Numpy-discussion] Why does fancy indexing work like this?

[Sebastian Berg]

On Thu, 2020-07-23 at 10:18 -0500, Sebastian Berg wrote:
> On Wed, 2020-07-22 at 17:35 -0600, Aaron Meurer wrote:
> > > About your warnings, do you have a nice way to do that?  The
> > > mechanism
> > > for warnings does not really give a good way to catch that a
> > > warning
> > > was raised and then turn it into an error.  Unless someone
> > > contributes
> > > a slick way to do it, I am not sure the complexity pays off.
> > 
> > I don't really know how flags and options and such work in NumPy,
> > but
> > I would imagine something like
> > 
> > if flags['post-deprecation'] = True: # Either a single flag for all
> > deprecations or a per-deprecation flag
> >     raise IndexError(...)
> > else:
> >     warnings.warn(...)
> > 
> 
> We have never done global flags for these things much in NumPy, I
> don't
> know of precedence in other packages, possibly aside future imports,
> but I am not even sure they have been used in this way.
> 
> > I don't know if the fact that the code that does this is in C
> > complicates things.
> > 
> > In other words, something that works kind of like __future__ flags
> > for
> > upgrading the behavior to post-deprecation.
> > 
> > > IIRC, I added the note about raising the warning, because in this
> > > particular case the deprecation warning (turned into an error)
> > > happens
> > > to be chained due to implementation details.  (so you do see the
> > > "original" error printed out).
> > 
> > Yes, it's nice that you can see it. But for my use case, I want to
> > be
> > able to "except IndexError". Basically, for ndindex, I test against
> > NumPy to make sure the semantics are identical, and that includes
> > making sure identical exceptions are raised. I also want to make it
> > so
> > that the ndindex semantics always follow post-deprecation behavior
> > for
> > any NumPy deprecations, since that leads to a cleaner API. But that
> > means that my test code has to do fancy shenanigans to catch these
> > deprecation warnings and treat them like the right errors.
> > 
> > But even as a general principle, I think for any deprecation
> > warning,
> > users should be able to update their code in such a way that the
> > current version doesn't give the warning and also it will continue
> > to
> 
> For FutureWarnings, I will always try very hard to give an option to
> opt-in to new behaviour or old behaviour – ideally with code
> compatible
> also with earlier NumPy versions.
> 
> Here, for a DeprecationWarning that has obviously no "alternative", I
> cannot think of any precedence in any other package or Python itself
> doing such a dance.  And it is extremely fringe (you only need it
> because you are testing another package against numpy behaviour!).
> 
> So I am happy to merge it if its proposed (maybe its easier for you
> to
> add this to NumPy then work around it in your tests), but I am
> honestly
> concerned that proposing this as a general principle is far more
> churn
> then worth the trouble.  At least unless there is some consensus (and
> probably precendence in the scientific python ecosystem or python
> itself).
> 

After writing this, I realized that I actually remember the *opposite*
discussion occurring before.  I think in some of the equality
deprecations, we actually raise the new error due to an internal
try/except clause.  And there was a complaint that its confusing that a
non-deprecation-warning is raised when the error will only happen with
DeprecationWarnings being set to error.

- Sebastian


> Cheers,
> 
> Sebastian
> 
> 
> > work and be idiomatic for future versions. For simple deprecations
> > where you remove a function x(), this is often as simple as telling
> > people to replace x() with y(). But these deprecations aren't so
> > simple, because the indexing itself is valid and will stay valid,
> > it's
> > just the behavior that will change. If there's no way to do this,
> > then
> > a deprecation warning serves little purpose because users who see
> > the
> > warning won't be able to do anything about it until things actually
> > change. There would be little difference from just changing things
> > outright. For the list as tuple indexing thing, you can already
> > kind
> > of do this by making sure your fancy indices are always arrays. For
> > the out of bounds one, it's a little harder. I guess for most
> > use-cases, you aren't actually checking for IndexErrors, and the
> > thing
> > that will become an error usually indicates a bug in user code, so
> > maybe it isn't a huge deal (I admit my use-cases aren't typical).
> > 
> > Aaron Meurer
> > _______________________________________________
> > NumPy-Discussion mailing list
> > NumPy-Discussion at python.org
> > https://mail.python.org/mailman/listinfo/numpy-discussion
> > 
> 
> _______________________________________________
> NumPy-Discussion mailing list
> NumPy-Discussion at python.org
> https://mail.python.org/mailman/listinfo/numpy-discussion

-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 833 bytes
Desc: This is a digitally signed message part
URL: <http://mail.python.org/pipermail/numpy-discussion/attachments/20200723/8cb350ee/attachment-0001.sig>