I don't understand why sometimes a direct assignment of a new dtype is possible (but messes up the values), and why at other times a seemingly harmless upcast (in my potentially ignorant point of view) is not possible. So, maybe a direct assignment of a new dtype is actually never a good idea? (I'm asking), and one should always go the route of newarray= array(oldarray, dtype=newdtype), but why then sometimes the upcast provides an error and forbids it and sometimes not? Examples: In [140]: slope.read_center_window() In [141]: slope.data.dtype Out[141]: dtype('float32') In [142]: slope.data[1,1] Out[142]: 10.044398 In [143]: val = slope.data[1,1] In [144]: slope.data.dtype='float64' In [145]: slope.data[1,1] Out[145]: 586.98938070189865 #----- #Here, the value of data[1,1] has completely changed (and so has the rest of the array), and no error was given. # But then... #---- In [146]: val.dtype Out[146]: dtype('float32') In [147]: val Out[147]: 10.044398 In [148]: val.dtype='float64' --------------------------------------------------------------------------- AttributeError Traceback (most recent call last) <ipython-input-148-52a373a41cac> in <module>() ----> 1 val.dtype='float64' AttributeError: attribute 'dtype' of 'numpy.generic' objects is not writable === end of code So why is there an error in the 2nd case, but no error in the first case? Is there a logic to it? Thanks, Michael
On Thu, Apr 18, 2013 at 7:31 PM, K.-Michael Aye <kmichael.aye@gmail.com>wrote:
I don't understand why sometimes a direct assignment of a new dtype is possible (but messes up the values), and why at other times a seemingly harmless upcast (in my potentially ignorant point of view) is not possible. So, maybe a direct assignment of a new dtype is actually never a good idea? (I'm asking), and one should always go the route of newarray= array(oldarray, dtype=newdtype), but why then sometimes the upcast provides an error and forbids it and sometimes not?
Examples:
In [140]: slope.read_center_window()
In [141]: slope.data.dtype Out[141]: dtype('float32')
In [142]: slope.data[1,1] Out[142]: 10.044398
In [143]: val = slope.data[1,1]
In [144]: slope.data.dtype='float64'
In [145]: slope.data[1,1] Out[145]: 586.98938070189865
#----- #Here, the value of data[1,1] has completely changed (and so has the rest of the array), and no error was given. # But then... #----
In [146]: val.dtype Out[146]: dtype('float32')
In [147]: val Out[147]: 10.044398
In [148]: val.dtype='float64' --------------------------------------------------------------------------- AttributeError Traceback (most recent call last) <ipython-input-148-52a373a41cac> in <module>() ----> 1 val.dtype='float64'
AttributeError: attribute 'dtype' of 'numpy.generic' objects is not writable
=== end of code
So why is there an error in the 2nd case, but no error in the first case? Is there a logic to it?
When you change a dtype like that in the first one, you aren't really upcasting anything. You are changing how numpy interprets the underlying bits. Because you went from a 32-bit element size to a 64-bit element size, you are actually seeing the double-precision representation of 2 of your original data points together. The correct way to cast is to do something like "a = slope.data.astype('float64')". That makes a copy and does the casting as safely as possible. As for the second one, you have what is called a numpy scalar. These aren't quite the same thing as a numpy array, and can be a bit more restrictive. Can you imagine what sort of issues that would pose if one could start viewing and modifying neighboring chunks of memory without ever having to mess around with pointers? It would be a hacker's dream! I hope that clears things up. Ben Root
On 2013-04-19 01:02:59 +0000, Benjamin Root said:
On Thu, Apr 18, 2013 at 7:31 PM, K.-Michael Aye <kmichael.aye@gmail.com> wrote: I don't understand why sometimes a direct assignment of a new dtype is possible (but messes up the values), and why at other times a seemingly harmless upcast (in my potentially ignorant point of view) is not possible. So, maybe a direct assignment of a new dtype is actually never a good idea? (I'm asking), and one should always go the route of newarray= array(oldarray, dtype=newdtype), but why then sometimes the upcast provides an error and forbids it and sometimes not?
Examples:
In [140]: slope.read_center_window()
In [141]: slope.data.dtype Out[141]: dtype('float32')
In [142]: slope.data[1,1] Out[142]: 10.044398
In [143]: val = slope.data[1,1]
In [144]: slope.data.dtype='float64'
In [145]: slope.data[1,1] Out[145]: 586.98938070189865
#----- #Here, the value of data[1,1] has completely changed (and so has the rest of the array), and no error was given. # But then... #----
In [146]: val.dtype Out[146]: dtype('float32')
In [147]: val Out[147]: 10.044398
In [148]: val.dtype='float64' --------------------------------------------------------------------------- AttributeError Traceback (most recent call last) <ipython-input-148-52a373a41cac> in <module>() ----> 1 val.dtype='float64'
AttributeError: attribute 'dtype' of 'numpy.generic' objects is not writable
=== end of code
So why is there an error in the 2nd case, but no error in the first case? Is there a logic to it?
When you change a dtype like that in the first one, you aren't really upcasting anything. You are changing how numpy interprets the underlying bits. Because you went from a 32-bit element size to a 64-bit element size, you are actually seeing the double-precision representation of 2 of your original data points together.
The correct way to cast is to do something like "a = slope.data.astype('float64')". That makes a copy and does the casting as safely as possible.
As for the second one, you have what is called a numpy scalar. These aren't quite the same thing as a numpy array, and can be a bit more restrictive. Can you imagine what sort of issues that would pose if one could start viewing and modifying neighboring chunks of memory without ever having to mess around with pointers? It would be a hacker's dream!
I hope that clears things up. Ben Root
yes, thanks! Michael
_______________________________________________ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
On Thu, Apr 18, 2013 at 10:04 PM, K.-Michael Aye <kmichael.aye@gmail.com> wrote:
On 2013-04-19 01:02:59 +0000, Benjamin Root said:
So why is there an error in the 2nd case, but no error in the first case? Is there a logic to it?
When you change a dtype like that in the first one, you aren't really upcasting anything. You are changing how numpy interprets the underlying bits. Because you went from a 32-bit element size to a 64-bit element size, you are actually seeing the double-precision representation of 2 of your original data points together.
I was wondering what would happen if there were not the right number of points available... In [225]: a = np.array((2.0, 3.0), dtype=np.float32) In [226]: a.dtype=np.float64 In [227]: a Out[227]: array([ 32.00000763]) OK , but: In [228]: a = np.array((2.0,), dtype=np.float32) In [229]: a.dtype=np.float64 --------------------------------------------------------------------------- ValueError Traceback (most recent call last) <ipython-input-229-0d494747aee1> in <module>() ----> 1 a.dtype=np.float64 ValueError: new type not compatible with array. so numpy is smart enough to not let you do it .. good thing. Final note -- changing the dtype in place like that is a very powerful and useful tool, but not likely to be used often -- it's really for things like working with odd binary data and the like. -Chris -- Christopher Barker, Ph.D. Oceanographer Emergency Response Division NOAA/NOS/OR&R (206) 526-6959 voice 7600 Sand Point Way NE (206) 526-6329 fax Seattle, WA 98115 (206) 526-6317 main reception Chris.Barker@noaa.gov
participants (3)
-
Benjamin Root -
Chris Barker - NOAA Federal -
K.-Michael Aye