better error message possible?
Hi All, Any reason why this:
import numpy numpy.zeros(10)[-123] Traceback (most recent call last): File "<stdin>", line 1, in <module> IndexError: index out of bounds
...could say this:
numpy.zeros(10)[-123] Traceback (most recent call last): File "<stdin>", line 1, in <module> IndexError: -123 is out of bounds
cheers, Chris -- Simplistix - Content Management, Batch Processing & Python Consulting - http://www.simplistix.co.uk
On Fri, Jun 1, 2012 at 10:46 AM, Chris Withers <chris@simplistix.co.uk> wrote:
Hi All,
Any reason why this:
>>> import numpy >>> numpy.zeros(10)[-123] Traceback (most recent call last): File "<stdin>", line 1, in <module> IndexError: index out of bounds
...could say this:
>>> numpy.zeros(10)[-123] Traceback (most recent call last): File "<stdin>", line 1, in <module> IndexError: -123 is out of bounds
Only that no-one has implemented it, I guess. If you want to then that'd be cool :-). To be generally useful for debugging, it would probably be good for the error message to also mention which dimension is involved, and/or the actual size of the array in that dimension. You can also get such error messages from expressions like 'arr[i, j, k]', after all, where it's even less obvious what went wrong. -- Nathaniel
On Fri, Jun 1, 2012 at 9:14 AM, Nathaniel Smith <njs@pobox.com> wrote:
On Fri, Jun 1, 2012 at 10:46 AM, Chris Withers <chris@simplistix.co.uk> wrote:
Hi All,
Any reason why this:
import numpy numpy.zeros(10)[-123] Traceback (most recent call last): File "<stdin>", line 1, in <module> IndexError: index out of bounds
...could say this:
numpy.zeros(10)[-123] Traceback (most recent call last): File "<stdin>", line 1, in <module> IndexError: -123 is out of bounds
Only that no-one has implemented it, I guess. If you want to then that'd be cool :-).
To be generally useful for debugging, it would probably be good for the error message to also mention which dimension is involved, and/or the actual size of the array in that dimension. You can also get such error messages from expressions like 'arr[i, j, k]', after all, where it's even less obvious what went wrong.
-- Nathaniel
+1, please! Ben Root
On Fri, Jun 1, 2012 at 10:46 AM, Chris Withers <chris@simplistix.co.uk>
Any reason why this:
>>> import numpy >>> numpy.zeros(10)[-123] Traceback (most recent call last): File "<stdin>", line 1, in <module> IndexError: index out of bounds
...could say this:
>>> numpy.zeros(10)[-123] Traceback (most recent call last): File "<stdin>", line 1, in <module> IndexError: -123 is out of bounds
Only that no-one has implemented it, I guess. If you want to then that'd be cool :-).
That would be nice, but to be fair, python itself doesn't do it either:
l = range(10) l[12] Traceback (most recent call last): File "<stdin>", line 1, in <module> IndexError: list index out of range
Though Python's standard error messages are lacking in a lot of places... -Chris -- Christopher Barker, Ph.D. Oceanographer Emergency Response Division NOAA/NOS/OR&R (206) 526-6959 voice 7600 Sand Point Way NE (206) 526-6329 fax Seattle, WA 98115 (206) 526-6317 main reception Chris.Barker@noaa.gov
On 01/06/2012 16:39, Benjamin Root wrote:
> >>> import numpy > >>> numpy.zeros(10)[-123] > Traceback (most recent call last): > File "<stdin>", line 1, in <module> > IndexError: index out of bounds > > ...could say this: > > >>> numpy.zeros(10)[-123] > Traceback (most recent call last): > File "<stdin>", line 1, in <module> > IndexError: -123 is out of bounds
Only that no-one has implemented it, I guess. If you want to then that'd be cool :-).
To be generally useful for debugging, it would probably be good for the error message to also mention which dimension is involved, and/or the actual size of the array in that dimension. You can also get such error messages from expressions like 'arr[i, j, k]', after all, where it's even less obvious what went wrong.
-- Nathaniel
+1, please!
Indeed, sadly I'm not a C developer. It's a pet bugbear of mine that Python's built-in exceptions often tell you what went wrong but not what data caused the error, even when it's easily to hand when raising the exception. Where's the right place to raise an issue that a numpy developer can hopefully make the (I suspect) simple change to get this behaviour? cheers, Chris -- Simplistix - Content Management, Batch Processing & Python Consulting - http://www.simplistix.co.uk
On Fri, Jun 1, 2012 at 10:56 AM, Chris Withers <chris@simplistix.co.uk>wrote:
On 01/06/2012 16:39, Benjamin Root wrote:
> >>> import numpy > >>> numpy.zeros(10)[-123] > Traceback (most recent call last): > File "<stdin>", line 1, in <module> > IndexError: index out of bounds > > ...could say this: > > >>> numpy.zeros(10)[-123] > Traceback (most recent call last): > File "<stdin>", line 1, in <module> > IndexError: -123 is out of bounds
Only that no-one has implemented it, I guess. If you want to then that'd be cool :-).
To be generally useful for debugging, it would probably be good for the error message to also mention which dimension is involved, and/or the actual size of the array in that dimension. You can also get such error messages from expressions like 'arr[i, j, k]', after all, where it's even less obvious what went wrong.
-- Nathaniel
+1, please!
Indeed, sadly I'm not a C developer. It's a pet bugbear of mine that Python's built-in exceptions often tell you what went wrong but not what data caused the error, even when it's easily to hand when raising the exception.
Where's the right place to raise an issue that a numpy developer can hopefully make the (I suspect) simple change to get this behaviour?
Hmm, how about I enable github issues for the numpy repository? It looks like we are headed that way and maybe now is a good time to get started. Chuck
On Fri, Jun 1, 2012 at 6:56 PM, Chris Withers <chris@simplistix.co.uk> wrote:
On 01/06/2012 16:39, Benjamin Root wrote:
> >>> import numpy > >>> numpy.zeros(10)[-123] > Traceback (most recent call last): > File "<stdin>", line 1, in <module> > IndexError: index out of bounds > > ...could say this: > > >>> numpy.zeros(10)[-123] > Traceback (most recent call last): > File "<stdin>", line 1, in <module> > IndexError: -123 is out of bounds
Only that no-one has implemented it, I guess. If you want to then that'd be cool :-).
To be generally useful for debugging, it would probably be good for the error message to also mention which dimension is involved, and/or the actual size of the array in that dimension. You can also get such error messages from expressions like 'arr[i, j, k]', after all, where it's even less obvious what went wrong.
-- Nathaniel
+1, please!
Indeed, sadly I'm not a C developer. It's a pet bugbear of mine that Python's built-in exceptions often tell you what went wrong but not what data caused the error, even when it's easily to hand when raising the exception.
I could look into this. There are only ~10 places the code generates this error, so it should be a pretty minor change. Ray Jones
On 4. juni 2012, at 16:27, Thouis (Ray) Jones wrote:
On Fri, Jun 1, 2012 at 6:56 PM, Chris Withers <chris@simplistix.co.uk> wrote:
On 01/06/2012 16:39, Benjamin Root wrote:
> >>> import numpy > >>> numpy.zeros(10)[-123] > Traceback (most recent call last): > File "<stdin>", line 1, in <module> > IndexError: index out of bounds > > ...could say this: > > >>> numpy.zeros(10)[-123] > Traceback (most recent call last): > File "<stdin>", line 1, in <module> > IndexError: -123 is out of bounds
Only that no-one has implemented it, I guess. If you want to then that'd be cool :-).
To be generally useful for debugging, it would probably be good for the error message to also mention which dimension is involved, and/or the actual size of the array in that dimension. You can also get such error messages from expressions like 'arr[i, j, k]', after all, where it's even less obvious what went wrong.
-- Nathaniel
+1, please!
Indeed, sadly I'm not a C developer. It's a pet bugbear of mine that Python's built-in exceptions often tell you what went wrong but not what data caused the error, even when it's easily to hand when raising the exception.
I could look into this. There are only ~10 places the code generates this error, so it should be a pretty minor change.
Ray Jones
Isn't it useful even if you change it in just one of those locations? Better to have the information available when you can, than to never have it. Paul
On Mon, Jun 4, 2012 at 4:27 PM, Thouis (Ray) Jones <thouis@gmail.com> wrote:
On Fri, Jun 1, 2012 at 6:56 PM, Chris Withers <chris@simplistix.co.uk> wrote:
On 01/06/2012 16:39, Benjamin Root wrote:
> >>> import numpy > >>> numpy.zeros(10)[-123] > Traceback (most recent call last): > File "<stdin>", line 1, in <module> > IndexError: index out of bounds > > ...could say this: > > >>> numpy.zeros(10)[-123] > Traceback (most recent call last): > File "<stdin>", line 1, in <module> > IndexError: -123 is out of bounds
Only that no-one has implemented it, I guess. If you want to then that'd be cool :-).
To be generally useful for debugging, it would probably be good for the error message to also mention which dimension is involved, and/or the actual size of the array in that dimension. You can also get such error messages from expressions like 'arr[i, j, k]', after all, where it's even less obvious what went wrong.
-- Nathaniel
+1, please!
Indeed, sadly I'm not a C developer. It's a pet bugbear of mine that Python's built-in exceptions often tell you what went wrong but not what data caused the error, even when it's easily to hand when raising the exception.
I could look into this. There are only ~10 places the code generates this error, so it should be a pretty minor change.
My initial estimate was low, but not overly so. An initial pass at adding index/dimension information to IndexErrors is here: https://github.com/thouis/numpy/tree/index_error_info A typical result:
numpy.zeros(3)[5] Traceback (most recent call last): File "<stdin>", line 1, in <module> IndexError: index 5 out of bounds in dimension 0
I thought it best to have erroring indices report their initial value:
numpy.zeros(3)[-15] Traceback (most recent call last): File "<stdin>", line 1, in <module> IndexError: index -15 out of bounds in dimension 0
This is different from some places in the code where IndexErrors already had bad index and dimension information (including the maximum value possible for an index in that dimension). I left these alone, though most of them would report that the bad index was -12 instead of -15. For instance: https://github.com/thouis/numpy/blob/index_error_info/numpy/core/src/multiar... Also there were a few indexing errors that were throwing ValueErrors. I changed these to IndexErrors. If someone could give this a cursory review before I issue a PR, I'd appreciate it. I don't expect that most of these code paths are heavily exercised in the tests (but I could be wrong). Ray Jones
On Mon, Jun 4, 2012 at 10:00 PM, Thouis (Ray) Jones <thouis@gmail.com> wrote:
On Mon, Jun 4, 2012 at 4:27 PM, Thouis (Ray) Jones <thouis@gmail.com> wrote:
I could look into this. There are only ~10 places the code generates this error, so it should be a pretty minor change.
My initial estimate was low, but not overly so. An initial pass at adding index/dimension information to IndexErrors is here: https://github.com/thouis/numpy/tree/index_error_info
Fabulous! I made a few comments there, but also:
A typical result:
numpy.zeros(3)[5] Traceback (most recent call last): File "<stdin>", line 1, in <module> IndexError: index 5 out of bounds in dimension 0
I would say "for", not "in". "index 5" is a bit ambiguous too... people might mis-read it as the dimension, like, "the 5th index value I gave"? Not sure how to make it unambiguous. Maybe: "IndexError: dimension 0 index out of bounds: got 5, size is 3" ?
I thought it best to have erroring indices report their initial value:
numpy.zeros(3)[-15] Traceback (most recent call last): File "<stdin>", line 1, in <module> IndexError: index -15 out of bounds in dimension 0
This is different from some places in the code where IndexErrors already had bad index and dimension information (including the maximum value possible for an index in that dimension). I left these alone, though most of them would report that the bad index was -12 instead of -15. For instance: https://github.com/thouis/numpy/blob/index_error_info/numpy/core/src/multiar...
I think this code you link to is actually correct, but yeah, it should definitely report whatever the user passed in, or it will be a debugging hindrance rather than a debugging help!
Also there were a few indexing errors that were throwing ValueErrors. I changed these to IndexErrors.
If someone could give this a cursory review before I issue a PR, I'd appreciate it. I don't expect that most of these code paths are heavily exercised in the tests (but I could be wrong).
Perhaps the easiest thing would be to just add a test? It should be about 1 line each per code path... or 2 if you check both the negative and positive versions. def test_index_bound_checking(): assert_raises(IndexError, my_array.__getitem__, (0, 100)) assert_raises(IndexError, my_array.__getitem__, (0, -101)) # etc. - N
On Mon, Jun 4, 2012 at 11:49 PM, Nathaniel Smith <njs@pobox.com> wrote:
On Mon, Jun 4, 2012 at 10:00 PM, Thouis (Ray) Jones <thouis@gmail.com> wrote:
On Mon, Jun 4, 2012 at 4:27 PM, Thouis (Ray) Jones <thouis@gmail.com> wrote:
I could look into this. There are only ~10 places the code generates this error, so it should be a pretty minor change.
My initial estimate was low, but not overly so. An initial pass at adding index/dimension information to IndexErrors is here: https://github.com/thouis/numpy/tree/index_error_info
Fabulous! I made a few comments there, but also:
A typical result:
numpy.zeros(3)[5] Traceback (most recent call last): File "<stdin>", line 1, in <module> IndexError: index 5 out of bounds in dimension 0
I would say "for", not "in".
"index 5" is a bit ambiguous too... people might mis-read it as the dimension, like, "the 5th index value I gave"? Not sure how to make it unambiguous. Maybe:
"IndexError: dimension 0 index out of bounds: got 5, size is 3"
?
How about: IndexError: 5 is out of bounds for dimension 0: must be in [-3, 3). to be maximally explicit about what values are allowed, and avoid the "index" confusion. Ray Jones
On Tue, Jun 5, 2012 at 12:15 PM, Thouis Jones <thouis.jones@curie.fr> wrote:
On Mon, Jun 4, 2012 at 11:49 PM, Nathaniel Smith <njs@pobox.com> wrote:
On Mon, Jun 4, 2012 at 10:00 PM, Thouis (Ray) Jones <thouis@gmail.com> wrote:
On Mon, Jun 4, 2012 at 4:27 PM, Thouis (Ray) Jones <thouis@gmail.com> wrote:
I could look into this. There are only ~10 places the code generates this error, so it should be a pretty minor change.
My initial estimate was low, but not overly so. An initial pass at adding index/dimension information to IndexErrors is here: https://github.com/thouis/numpy/tree/index_error_info
Fabulous! I made a few comments there, but also:
A typical result:
numpy.zeros(3)[5] Traceback (most recent call last): File "<stdin>", line 1, in <module> IndexError: index 5 out of bounds in dimension 0
I would say "for", not "in".
"index 5" is a bit ambiguous too... people might mis-read it as the dimension, like, "the 5th index value I gave"? Not sure how to make it unambiguous. Maybe:
"IndexError: dimension 0 index out of bounds: got 5, size is 3"
?
How about: IndexError: 5 is out of bounds for dimension 0: must be in [-3, 3).
to be maximally explicit about what values are allowed, and avoid the "index" confusion.
Or perhaps "axis" instead of "dimension", since this is how they are referred to in most numpy argument lists.
I've opened a PR at https://github.com/numpy/numpy/pull/296 for discussion. A typical result
np.zeros((3,3))[[1,2,3]] Traceback (most recent call last): File "<stdin>", line 1, in <module> IndexError: index 3 is out of bounds for axis 0: [-3,3)
Ray Jones
On 7. juni 2012, at 10:30, Thouis (Ray) Jones wrote:
I've opened a PR at https://github.com/numpy/numpy/pull/296 for discussion.
A typical result
np.zeros((3,3))[[1,2,3]] Traceback (most recent call last): File "<stdin>", line 1, in <module> IndexError: index 3 is out of bounds for axis 0: [-3,3)
Ray Jones _______________________________________________ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
I would prefer: IndexError: index 3 is out of bounds for axis 0: [-3,2] as I find the 3) notation a bit weird - after all, indices are not floats, so 2.999 or 2.3 doesn't make sense as an index. An alternative is to not refer to negative indices and say IndexError: index 3 is out of bounds for axis 0, shape was: (3,) (print more axes when the number of axes is higher.) BTW, I'm really glad someone is taking an interest in these error messages, it's a great idea! Paul
Paul Anton Letnes <paul.anton.letnes <at> gmail.com> writes:
I would prefer: IndexError: index 3 is out of bounds for axis 0: [-3,2] as I find the 3) notation a bit weird - after all, indices are not floats, so 2.999 or 2.3 doesn't make sense as an index.
An alternative is to not refer to negative indices and say IndexError: index 3 is out of bounds for axis 0, shape was: (3,) (print more axes when the number of axes is higher.)
+1 for the latter suggestion - if the array shape is available it's a great help in debugging the error. An alternative wording could be: IndexError: Index 3 is out of bounds for axis 0 of an array of shape (3,2,4) -Dave
On Thu, Jun 7, 2012 at 11:44 AM, Dave Hirschfeld <dave.hirschfeld@gmail.com> wrote:
Paul Anton Letnes <paul.anton.letnes <at> gmail.com> writes:
I would prefer: IndexError: index 3 is out of bounds for axis 0: [-3,2] as I find the 3) notation a bit weird - after all, indices are not floats, so 2.999 or 2.3 doesn't make sense as an index.
An alternative is to not refer to negative indices and say IndexError: index 3 is out of bounds for axis 0, shape was: (3,) (print more axes when the number of axes is higher.)
+1 for the latter suggestion - if the array shape is available it's a great help in debugging the error.
I agree that reporting the shape would be good, but it's usually not available at the point that the indices are found to be out-of-bounds, due to (often implicit) flattening. I think it might be possible to track and report that the array was flattened, which might help avoid some confusion when the maximum index reported in the Exception doesn't match any of the dimensions of the array being indexed due to flattening. Another possibility I entertained was to split the too-high vs. too-low cases: IndexError: index 3 is out of bounds for axis 0: must be < 3 IndexError: index -4 is out of bounds for axis 0: must be >= -3 Ray Jones
On Jun 7, 2012, at 4:32 AM, Paul Anton Letnes wrote:
On 7. juni 2012, at 10:30, Thouis (Ray) Jones wrote:
I've opened a PR at https://github.com/numpy/numpy/pull/296 for discussion.
A typical result
np.zeros((3,3))[[1,2,3]] Traceback (most recent call last): File "<stdin>", line 1, in <module> IndexError: index 3 is out of bounds for axis 0: [-3,3)
Ray Jones _______________________________________________ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
I would prefer: IndexError: index 3 is out of bounds for axis 0: [-3,2] as I find the 3) notation a bit weird - after all, indices are not floats, so 2.999 or 2.3 doesn't make sense as an index.
Actually, with slicing, you can attempt indexing using floats (as referred to here: http://stackoverflow.com/questions/8514547/why-ndarray-allow-floating-point-...), so I disagree with the integer notation [-3,2] I actually prefer the subsequent suggestion of reporting the shape, since that's what I always check when debugging index errors anyway -- having it in the error message would save me a ton of time.
An alternative is to not refer to negative indices and say IndexError: index 3 is out of bounds for axis 0, shape was: (3,) (print more axes when the number of axes is higher.)
BTW, I'm really glad someone is taking an interest in these error messages, it's a great idea!
… an enthusiastic +1!
Paul
participants (10)
-
Benjamin Root -
Charles R Harris -
Chris Barker -
Chris Withers -
Dave Hirschfeld -
Nathaniel Smith -
Paul Anton Letnes -
Thouis (Ray) Jones -
Thouis Jones -
Travis Vaught