Is indexing with floats really allowed in numpy?
a=numpy.array([1,2,3,4]) a[2.99] 3
Nadia Dencheva
Looks like it truncates to an int. But I wouldn't do it especially if you use floating operations (+,*,/, etc.) since what you get may not truncate to the integer you expect. Remember floats are approximate representations for many rational numbers. Stick with integers, IMHO. --- Nadia Dencheva <dencheva@stsci.edu> wrote:
Is indexing with floats really allowed in numpy?
a=numpy.array([1,2,3,4]) a[2.99] 3
-- Lou Pecora, my views are my own. ____________________________________________________________________________________ Be a better Globetrotter. Get better travel answers from someone who knows. Yahoo! Answers - Check it out. http://answers.yahoo.com/dir/?link=list&sid=396545469
This should return an error and not silently truncate to int. Nadia Lou Pecora wrote:
Looks like it truncates to an int. But I wouldn't do it especially if you use floating operations (+,*,/, etc.) since what you get may not truncate to the integer you expect. Remember floats are approximate representations for many rational numbers. Stick with integers, IMHO.
--- Nadia Dencheva <dencheva@stsci.edu> wrote:
Is indexing with floats really allowed in numpy?
a=numpy.array([1,2,3,4]) a[2.99] 3
-- Lou Pecora, my views are my own.
____________________________________________________________________________________ Be a better Globetrotter. Get better travel answers from someone who knows. Yahoo! Answers - Check it out. http://answers.yahoo.com/dir/?link=list&sid=396545469 _______________________________________________ Numpy-discussion mailing list Numpy-discussion@scipy.org http://projects.scipy.org/mailman/listinfo/numpy-discussion
On Fri, Sep 28, 2007 at 03:07:30PM -0400, Nadia Dencheva wrote:
This should return an error and not silently truncate to int.
Why do you say that? The current behaviour is consistent and well defined: a[x] == a[int(x)] We certainly can't change it now (just imagine all the code out there that will break); but I personally don't think it is a big problem. On a somewhat related note, you may also be interested in the PEP at http://docs.python.org/whatsnew/pep-357.html Regards Stéfan
On Sep 28, 2007, at 4:23 PM, Stefan van der Walt wrote:
On Fri, Sep 28, 2007 at 03:07:30PM -0400, Nadia Dencheva wrote:
This should return an error and not silently truncate to int.
Why do you say that? The current behaviour is consistent and well defined:
a[x] == a[int(x)]
I disagree. It is neither consistent nor well defined. It is not consistent with Python list indexing behavior. It is not consistent with numpy behavior:
x = arange(10) x[array(2.99)]
raises an exception It didn't work in Numeric It didn't work in numarray
We certainly can't change it now (just imagine all the code out there that will break); but I personally don't think it is a big problem.
I disagree. If people are willing to change the Class API of numpy to be consistent with Python, they certainly should be willing to change this. This behavior is new with numpy, so there should not be a lot of code that depends on it (and shame on those that do :-).
On a somewhat related note, you may also be interested in the PEP at
I believe that this PEP is irrelevant. It's purpose is to allow numpy rank-0 arrays (what numpy returns on indexing single values in arrays) to be used as an index in Python lists. It does not suggest that float values should be used as indices for sequences. It does provide a mechanism so that numpy rank-0 float arrays could be used as indices to lists, but doesn't mean that they should. And currently, they don't work as indices to lists (try it). So I can't see any reason not to treat the current behavior as undesirable. It is not long standing (it wasn't available in numpy, and only in odd ways in numarray--and it was a bug there [I feel I can say that without contradiction :-)]. Now that the compatibility can of worms is open, fixing this is a small thing compared to the other issue being raised. Perry Greenfield
On 10/3/07, Perry Greenfield <perry@stsci.edu> wrote:
On Sep 28, 2007, at 4:23 PM, Stefan van der Walt wrote:
On Fri, Sep 28, 2007 at 03:07:30PM -0400, Nadia Dencheva wrote:
This should return an error and not silently truncate to int.
Why do you say that? The current behaviour is consistent and well defined:
a[x] == a[int(x)]
I disagree. It is neither consistent nor well defined.
It is not consistent with Python list indexing behavior.
It is not consistent with numpy behavior:
x = arange(10) x[array(2.99)]
raises an exception
It didn't work in Numeric It didn't work in numarray
We certainly can't change it now (just imagine all the code out there that will break); but I personally don't think it is a big problem.
I disagree. If people are willing to change the Class API of numpy to be consistent with Python, they certainly should be willing to change this. This behavior is new with numpy, so there should not be a lot of code that depends on it (and shame on those that do :-).
On a somewhat related note, you may also be interested in the PEP at
I believe that this PEP is irrelevant. It's purpose is to allow numpy rank-0 arrays (what numpy returns on indexing single values in arrays) to be used as an index in Python lists. It does not suggest that float values should be used as indices for sequences. It does provide a mechanism so that numpy rank-0 float arrays could be used as indices to lists, but doesn't mean that they should. And currently, they don't work as indices to lists (try it).
So I can't see any reason not to treat the current behavior as undesirable. It is not long standing (it wasn't available in numpy, and only in odd ways in numarray--and it was a bug there [I feel I can say that without contradiction :-)]. Now that the compatibility can of worms is open, fixing this is a small thing compared to the other issue being raised.
I agree that this should be changed, and the sooner the better. Look at how many problems arange with float arguments causes, and the often expressed wish that it had never allowed floats in the first place. I suspect using floats as indexes is almost always a bug, and where it isn't the code would be much clearer if the conversion to integer was made explicit. Chuck
On Wed, Oct 03, 2007 at 11:12:07AM -0400, Perry Greenfield wrote:
On Sep 28, 2007, at 4:23 PM, Stefan van der Walt wrote:
On Fri, Sep 28, 2007 at 03:07:30PM -0400, Nadia Dencheva wrote:
This should return an error and not silently truncate to int.
Why do you say that? The current behaviour is consistent and well defined:
a[x] == a[int(x)]
I disagree. It is neither consistent nor well defined.
It works for other objects too: class MyIndex(object): def __int__(self): return 1 m = M() x = N.array([1,2,3] x[m] == x[1]
It is not consistent with Python list indexing behavior.
Neither are most other forms of ndarray indexing: x = [1,2,3] x[[1,2]] yields an error.
It is not consistent with numpy behavior:
x = arange(10) x[array(2.99)]
raises an exception
That seems to be an exception to the rule. I agree that both x[2.99] and x[array(2.99)] should behave the same way.
We certainly can't change it now (just imagine all the code out there that will break); but I personally don't think it is a big problem.
I disagree. If people are willing to change the Class API of numpy to be consistent with Python, they certainly should be willing to change this. This behavior is new with numpy, so there should not be a lot of code that depends on it (and shame on those that do :-).
Let me rephrase: we cannot change the API until 1.1, unless this is seen as a bug. To which other API changes are you referring? The style changes is a different matter entirely. My point was that the current behaviour is easy to predict, but I am not especially partial on the matter. Regards Stéfan
On Oct 3, 2007, at 1:35 PM, Stefan van der Walt wrote:
We certainly can't change it now (just imagine all the code out there that will break); but I personally don't think it is a big problem.
I disagree. If people are willing to change the Class API of numpy to be consistent with Python, they certainly should be willing to change this. This behavior is new with numpy, so there should not be a lot of code that depends on it (and shame on those that do :-).
Let me rephrase: we cannot change the API until 1.1, unless this is seen as a bug. To which other API changes are you referring? The style changes is a different matter entirely.
The recent numpy and scipy threads on "adopting Python Style Guide for classes". If this is what you mean by style changes and it ends up changing array to Array and int32 to Int32, it's not just a style change, it will break lots of code ("Since it should effect the user API..."). These are API changes even if you consider case of the class name just style. Perry Greenfield
On Wed, Oct 03, 2007 at 01:50:01PM -0400, Perry Greenfield wrote:
Let me rephrase: we cannot change the API until 1.1, unless this is seen as a bug. To which other API changes are you referring? The style changes is a different matter entirely.
The recent numpy and scipy threads on "adopting Python Style Guide for classes". If this is what you mean by style changes and it ends up changing array to Array and int32 to Int32, it's not just a style change, it will break lots of code ("Since it should effect the user API..."). These are API changes even if you consider case of the class name just style.
Don't worry, nothing is being rushed through. Only internal representations are currently being changed (e.g. unit tests). No changes to the API will be made until 1.1, and then only after the discussion is over. I think your quote above refers to the e-mail Jarrod Millman wrote, but that was simply a typo, and should have read "since it shouldn't affect the user API". Regards Stéfan
Stefan van der Walt wrote:
The current behavior is consistent and well
defined: a[x] == a[int(x)]
This is all possible because of PEP 357: http://www.python.org/dev/peps/pep-0357/ However, when I read this from the PEP: """ It is not possible to use the nb_int (and __int__ special method) for this purpose because that method is used to *coerce* objects to integers. It would be inappropriate to allow every object that can be coerced to an integer to be used as an integer everywhere Python expects a true integer. For example, if __int__ were used to convert an object to an integer in slicing, then float objects would be allowed in slicing and x[3.2:5.8] would not raise an error as it should. """ It seems pretty clear that only integer types were intended to by used as indexes. Does that make this a bug? I'll defer that to others more in the know than I. -Chris -- Christopher Barker, Ph.D. Oceanographer Emergency Response Division NOAA/NOS/OR&R (206) 526-6959 voice 7600 Sand Point Way NE (206) 526-6329 fax Seattle, WA 98115 (206) 526-6317 main reception Chris.Barker@noaa.gov
On 10/3/07, Christopher Barker <Chris.Barker@noaa.gov> wrote:
Stefan van der Walt wrote:
The current behavior is consistent and well
defined: a[x] == a[int(x)]
This is all possible because of PEP 357:
I think that the current behavior has always been possible; arbitrary objects can be passed as indices and coercing to int was always possible. http://www.python.org/dev/peps/pep-0357/
However, when I read this from the PEP:
""" It is not possible to use the nb_int (and __int__ special method) for this purpose because that method is used to *coerce* objects to integers. It would be inappropriate to allow every object that can be coerced to an integer to be used as an integer everywhere Python expects a true integer. For example, if __int__ were used to convert an object to an integer in slicing, then float objects would be allowed in slicing and x[3.2:5.8] would not raise an error as it should. """
It seems pretty clear that only integer types were intended to by used as indexes. Does that make this a bug? I'll defer that to others more in the know than I.
As I understand it, the whole point of PEP-357 was to allow the coercion of int-like things (numpy.int32 say, or your own private integerish class) to be used as indices without also allowing things that aren't integers, but that can be coerced to integers (floats for instance) to slip through. So yes, I'd say this is a bug. Probably someplace that should be using __index__ or the C equivalent is still using __int__, but I haven't had time to look. On the other hand, this may well be a bug that people rely on and it's not doing drastic immediate harm. So, while I think it should get fixed eventually, it's probably fine to wait till 1.1 or whenever the next convenient time is. -- . __ . |-\ . . tim.hochberg@ieee.org
Timothy Hochberg wrote:
As I understand it, the whole point of PEP-357 was to allow the coercion of int-like things (numpy.int32 say, or your own private integerish class) to be used as indices without also allowing things that aren't integers, but that can be coerced to integers (floats for instance) to slip through. So yes, I'd say this is a bug. Probably someplace that should be using __index__ or the C equivalent is still using __int__, but I haven't had time to look.
I believe the problem is in numpy/core/src/arrayobject.c: 2635 static PyObject * 2636 array_subscript_simple(PyArrayObject *self, PyObject *op) 2637 { 2638 intp dimensions[MAX_DIMS], strides[MAX_DIMS]; 2639 intp offset; 2640 int nd; 2641 PyArrayObject *other; 2642 intp value; 2643 2644 value = PyArray_PyIntAsIntp(op); -- Robert Kern "I have come to believe that the whole world is an enigma, a harmless enigma that is made terrible by our own mad attempt to interpret it as though it had an underlying truth." -- Umberto Eco
On 10/3/07, Timothy Hochberg <tim.hochberg@ieee.org> wrote:
As I understand it, the whole point of PEP-357 was to allow the coercion of int-like things (numpy.int32 say, or your own private integerish class) to be used as indices without also allowing things that aren't integers, but that can be coerced to integers (floats for instance) to slip through. So yes, I'd say this is a bug. Probably someplace that should be using __index__ or the C equivalent is still using __int__, but I haven't had time to look.
It seems like a bug to me as well.
On the other hand, this may well be a bug that people rely on and it's not doing drastic immediate harm. So, while I think it should get fixed eventually, it's probably fine to wait till 1.1 or whenever the next convenient time is.
I don't know that it is essential to fix in the 1.0.x series, but 1.1 is still several months away. And the longer this "bug" is out there, the greater the chance that people will write code that depends on it. So I would tentatively vote to fix it sooner rather than later. -- Jarrod Millman Computational Infrastructure for Research Labs 10 Giannini Hall, UC Berkeley phone: 510.643.4014 http://cirl.berkeley.edu/
participants (9)
-
Charles R Harris -
Christopher Barker -
Jarrod Millman -
Lou Pecora -
Nadia Dencheva -
Perry Greenfield -
Robert Kern -
Stefan van der Walt -
Timothy Hochberg