Re: [Numpy-discussion] A couple more Numeric incompatibilitiesand a possible bug
cos(1) 0.54030227661132813
gives a different result to cos(1.) <snip> Do others think precision is being lost unnecessarily?
No. Do you have any suggestions?
Yes. My problem is that numarray is not replicating the behaviour of the Python math module, which presumably coerces immediately to the float precision of the underlying platform. This is probably a float64 in Windows. If you don't explicitly specify that float32 is to be used, shouldn't the default be for a rank-0 value to immediately coerce ints to the same precision as the native Python float type on the underlying platform, since you know it will be coerced to that later and in the meantime you've lost precision because you've applied a function to a value of lower precision?
Finally, has anyone got any comment about whether the __repr__ versus __str__ display of object array members is the expected behaviour?
Yes. The basic look is exactly what I wanted:
a = numarray.objects.fromlist(range(10)) str(a) '[0, 1, 2, 3, 4, 5, 6, 7, 8, 9]' repr(a) 'ObjectArray([0, 1, 2, 3, 4, 5, 6, 7, 8, 9])'
But I'm open to suggestions and recognize that the handling of spaces/item sizing is weak (although it's not pronounced in this simple example). Do people want something else?
Regards, Todd
Your example doesn't demonstrate the problem I'm talking about because for integers the __str__ and __repr__ methods are identical. However, it does show a difference between Numeric and numarray, which would display your str(a) as '[0 1 2 3 4 5 6 7 8 9 ]' If you make an ObjectArray containing objects whose __repr__ and __str__ methods are not the same, I think you need a way of displaying the contained objects via their type-specific __str__ method. Numeric did this by default, whereas numarray has changed that behaviour. I'll go back to my example again. In my example str() does not display the array elements in the same way that Numeric does. In my case, numarray does this:
print ArrayOfErr([909., 802., 677., 585.], 1.0) [Err(909.0,1.0,1.0), Err(802.0,1.0,1.0), Err(677.0,1.0,1.0), Err(585.0,1.0,1.0)]
print str(ArrayOfErr([909., 802., 677., 585.], 1.0)) [Err(909.0,1.0,1.0), Err(802.0,1.0,1.0), Err(677.0,1.0,1.0), Err(585.0,1.0,1.0)]
print repr(ArrayOfErr([909., 802., 677., 585.], 1.0)) ObjectArray([Err(909.0,1.0,1.0), Err(802.0,1.0,1.0), Err(677.0,1.0,1.0), Err(585.0,1.0,1.0)])
ie. str() is not using the __str__ method I have defined for my Err class. Numeric correctly uses my class's Err.__str__ method and does this instead:
print ArrayOfErr([909., 802., 677., 585.], 1.0) [909.0 +1.0/-1.0 802.0 +1.0/-1.0 677.0 +1.0/-1.0 585.0 +1.0/-1.0 ]
In summary, I think te behaviour should be
print str(ArrayOfErr([909., 802., 677., 585.], 1.0)) [909.0 +1.0/-1.0 802.0 +1.0/-1.0 677.0 +1.0/-1.0 585.0 +1.0/-1.0 ]
print repr(ArrayOfErr([909., 802., 677., 585.], 1.0)) ObjectArray([Err(909.0,1.0,1.0), Err(802.0,1.0,1.0), Err(677.0,1.0,1.0), Err(585.0,1.0,1.0)])
I hope I've clarified what I'm getting at, although I probably haven't :-) Gary -- ___________________________________________________________ Sign-up for Ads Free at Mail.com http://promo.mail.com/adsfreejump.htm
On Sun, 2004-03-28 at 22:46, Gary Ruben wrote:
cos(1) 0.54030227661132813
gives a different result to cos(1.) <snip> Do others think precision is being lost unnecessarily?
No. Do you have any suggestions?
Yes. My problem is that numarray is not replicating the behaviour of the Python math module, which presumably coerces immediately to the float precision of the underlying platform. This is probably a float64 in Windows. If you don't explicitly specify that float32 is to be used, shouldn't the default be for a rank-0 value to immediately coerce ints to the same precision as the native Python float type on the underlying platform, since you know it will be coerced to that later and in the meantime you've lost precision because you've applied a function to a value of lower precision?
I see your point and I'll talk it over with Perry unless he chimes in on his own. I'm ambivalent: while it would be nice for the numarray ufuncs to act as full equivalents to the Python math functions in scalar cases, numarray has to strike a balance between managing storage space and maintaining precision and these two goals are in conflict. I think using an array package you have to be at least a little more aware of storage concerns.
Finally, has anyone got any comment about whether the __repr__ versus __str__ display of object array members is the expected behaviour?
Yes. The basic look is exactly what I wanted:
Don't ask how, but I just noticed that my foot has nearly the same dimensions of my mouth.
a = numarray.objects.fromlist(range(10)) str(a) '[0, 1, 2, 3, 4, 5, 6, 7, 8, 9]' repr(a) 'ObjectArray([0, 1, 2, 3, 4, 5, 6, 7, 8, 9])'
But I'm open to suggestions and recognize that the handling of spaces/item sizing is weak (although it's not pronounced in this simple example). Do people want something else?
Regards, Todd
Your example doesn't demonstrate the problem I'm talking about because for integers the __str__ and __repr__ methods are identical. However, it does show a difference between Numeric and numarray, which would display your str(a) as '[0 1 2 3 4 5 6 7 8 9 ]'
Well, almost what I wanted. This will change.
If you make an ObjectArray containing objects whose __repr__ and __str__ methods are not the same, I think you need a way of displaying the contained objects via their type-specific __str__ method. Numeric did this by default, whereas numarray has changed that behaviour.
This will change too.
I'll go back to my example again. In my example str() does not display the array elements in the same way that Numeric does. In my case, numarray does this:
print ArrayOfErr([909., 802., 677., 585.], 1.0) [Err(909.0,1.0,1.0), Err(802.0,1.0,1.0), Err(677.0,1.0,1.0), Err(585.0,1.0,1.0)]
print str(ArrayOfErr([909., 802., 677., 585.], 1.0)) [Err(909.0,1.0,1.0), Err(802.0,1.0,1.0), Err(677.0,1.0,1.0), Err(585.0,1.0,1.0)]
print repr(ArrayOfErr([909., 802., 677., 585.], 1.0)) ObjectArray([Err(909.0,1.0,1.0), Err(802.0,1.0,1.0), Err(677.0,1.0,1.0), Err(585.0,1.0,1.0)])
ie. str() is not using the __str__ method I have defined for my Err class. Numeric correctly uses my class's Err.__str__ method and does this instead:
print ArrayOfErr([909., 802., 677., 585.], 1.0) [909.0 +1.0/-1.0 802.0 +1.0/-1.0 677.0 +1.0/-1.0 585.0 +1.0/-1.0 ]
In summary, I think te behaviour should be
print str(ArrayOfErr([909., 802., 677., 585.], 1.0)) [909.0 +1.0/-1.0 802.0 +1.0/-1.0 677.0 +1.0/-1.0 585.0 +1.0/-1.0 ]
print repr(ArrayOfErr([909., 802., 677., 585.], 1.0)) ObjectArray([Err(909.0,1.0,1.0), Err(802.0,1.0,1.0), Err(677.0,1.0,1.0), Err(585.0,1.0,1.0)])
I hope I've clarified what I'm getting at, although I probably haven't :-)
Quite clear. Stuff will change. Regards, Todd -- Todd Miller <jmiller@stsci.edu>
On Mon, Mar 29, 2004 at 06:55:57AM -0500, Todd Miller wrote:
On Sun, 2004-03-28 at 22:46, Gary Ruben wrote:
> cos(1) 0.54030227661132813
gives a different result to cos(1.) <snip> Do others think precision is being lost unnecessarily?
No. Do you have any suggestions?
Yes. My problem is that numarray is not replicating the behaviour of the Python math module, which presumably coerces immediately to the float precision of the underlying platform. This is probably a float64 in Windows. If you don't explicitly specify that float32 is to be used, shouldn't the default be for a rank-0 value to immediately coerce ints to the same precision as the native Python float type on the underlying platform, since you know it will be coerced to that later and in the meantime you've lost precision because you've applied a function to a value of lower precision?
I see your point and I'll talk it over with Perry unless he chimes in on his own. I'm ambivalent: while it would be nice for the numarray ufuncs to act as full equivalents to the Python math functions in scalar cases, numarray has to strike a balance between managing storage space and maintaining precision and these two goals are in conflict. I think using an array package you have to be at least a little more aware of storage concerns.
I'll throw in a vote for 'cos(1)' coercing 1 to a (equivalent-to-) Python float. The reason is that I have 'from numarray import *' for my interactive interpreter, as I use numarray a lot in that. I would rather not have to remember to always cast my args to ufuncs to arrays. This isn't a storage space problem, per se., the question is whether cos(1) and cos(1.0) should return the same answer. Having cos(1) cast 1 to a Float32 makes numarray much less useful as a calculator -- mistakes are more likely. I'll probably switch my interpreter back to Numeric so as not to make mistakes, or I'll write a wrapper (which I'll post if I do). Whether cos([1,1,1]) and cos([1.0,1.0,1.0]) are the same I'd say is different, as there I've taken the conscious decision not to cast to an array. (And the fact that cos([1,1,1]) will return an array printed out with 'type=Float32' is a clue to what I've done.) -- |>|\/|< /--------------------------------------------------------------------------\ |David M. Cooke http://arbutus.physics.mcmaster.ca/dmc/ |cookedm@physics.mcmaster.ca
David Cooke wrote:
On Mon, Mar 29, 2004 at 06:55:57AM -0500, Todd Miller wrote:
On Sun, 2004-03-28 at 22:46, Gary Ruben wrote:
>> cos(1) 0.54030227661132813
gives a different result to cos(1.) <snip> Do others think precision is being lost unnecessarily?
No. Do you have any suggestions?
Yes. My problem is that numarray is not replicating the behaviour of the Python math module, which presumably coerces immediately to the float precision of the underlying platform. This is probably a float64 in Windows. If you don't explicitly specify that float32 is to be used, shouldn't the default be for a rank-0 value to immediately coerce ints to the same precision as the native Python float type on the underlying platform, since you know it will be coerced to that later and in the meantime you've lost precision because you've applied a function to a value of lower precision?
I see your point and I'll talk it over with Perry unless he chimes in on his own. I'm ambivalent: while it would be nice for the numarray ufuncs to act as full equivalents to the Python math functions in scalar cases, numarray has to strike a balance between managing storage space and maintaining precision and these two goals are in conflict. I think using an array package you have to be at least a little more aware of storage concerns.
I'll throw in a vote for 'cos(1)' coercing 1 to a (equivalent-to-) Python float. The reason is that I have 'from numarray import *' for my interactive interpreter, as I use numarray a lot in that. I would rather not have to remember to always cast my args to ufuncs to arrays. This isn't a storage space problem, per se., the question is whether cos(1) and cos(1.0) should return the same answer.
Having cos(1) cast 1 to a Float32 makes numarray much less useful as a calculator -- mistakes are more likely. I'll probably switch my interpreter back to Numeric so as not to make mistakes, or I'll write a wrapper (which I'll post if I do).
Whether cos([1,1,1]) and cos([1.0,1.0,1.0]) are the same I'd say is different, as there I've taken the conscious decision not to cast to an array. (And the fact that cos([1,1,1]) will return an array printed out with 'type=Float32' is a clue to what I've done.)
Let me say that I see the last issue (whether cos([1,1,1]) and cos([1.0,1.0,1.0]) are the same) is important. If cos([1,1,1])[0] != cos(1) I think that is just as bad. So the question is whether ints scalars and arrays should automatically be promoted to double floats when given to unary scientific functions (sqrt, sin, cos, log, etc.). By the way, if an Int32 is so promoted, I don't see why it isn't the same for Int8, Int16, etc. But it appears that Numeric promotes Int32 to Float64 for cos(), but promotes In16 to Float32 for cos(). I believe making this change to numarray is fairly easy (we'd have to check). But is what most people want? Should we retain complete compatibility with Numeric (which, as pointed out, treats different kinds of integers differently) Just to summarize what I see as acceptable choices: 1) ints promote to Float32 (current behavior) 2) ints promote to Float64 3) Int32 promotes to Float64, lesser ints promote to Float32 (Numeric behavior) Perry
On Mar 31, 2004, at 7:39 AM, Perry Greenfield wrote:
Just to summarize what I see as acceptable choices:
1) ints promote to Float32 (current behavior) 2) ints promote to Float64 3) Int32 promotes to Float64, lesser ints promote to Float32 (Numeric behavior)
+1 on (2) I want math.cos(x) to give the same answer as numarray.cos( array( (x) ) ) as much as possible. I like that Numarray does not do as much auto-upcasting as Numeric, but if I want to control the size of my data arrays, I'm not going to be passing an int into a floating point function. i.e, I can always do: cos( array( (x), Float32 ) ) Then I would want the result to be a Float32 -Chris Christopher Barker, Ph.D. Oceanographer NOAA/OR&R/HAZMAT (206) 526-6959 voice 7600 Sand Point Way NE (206) 526-6329 fax Seattle, WA 98115 (206) 526-6317 main reception Chris.Barker@noaa.gov
Chris Barker wrote:
On Mar 31, 2004, at 7:39 AM, Perry Greenfield wrote:
Just to summarize what I see as acceptable choices:
1) ints promote to Float32 (current behavior) 2) ints promote to Float64 3) Int32 promotes to Float64, lesser ints promote to Float32 (Numeric behavior)
+1 on (2)
I want math.cos(x) to give the same answer as numarray.cos( array( (x) ) ) as much as possible. I like that Numarray does not do as much auto-upcasting as Numeric, but if I want to control the size of my data arrays, I'm not going to be passing an int into a floating point function. i.e, I can always do:
cos( array( (x), Float32 ) )
Then I would want the result to be a Float32
+1 on (2) as well, based on a similar reasoning to Chris (who saved me the time of typing it up cleanly :) Cheers, f
On Wed, 31 Mar 2004, Perry Greenfield wrote:
Just to summarize what I see as acceptable choices:
1) ints promote to Float32 (current behavior) 2) ints promote to Float64 3) Int32 promotes to Float64, lesser ints promote to Float32 (Numeric behavior)
I'm also +1 on option (2). I believe that this will not cause problems with unwanted promotions. The result of cos(1) is a scalar Python float, regardless of whether 1 is converted to a Float32 or a Float64 before the cosine is evaluated. Then if that scalar is used with a Float32 numarray, it gets converted to a Float32 and the result is Float32. For example:
from numarray import * cos(1)*ones(1,Float32) array([ 0.54030228], type=Float32) cos(1.0)*ones(1,Float32) array([ 0.54030228], type=Float32)
So it seems to me there is no downside to converting integers to Float64 when the ufunc requires a float. And I find it hard to believe that any Numeric users are counting on behavior (3). Rick
Rick White wrote:
On Wed, 31 Mar 2004, Perry Greenfield wrote:
Just to summarize what I see as acceptable choices:
1) ints promote to Float32 (current behavior) 2) ints promote to Float64 3) Int32 promotes to Float64, lesser ints promote to Float32 (Numeric behavior)
I'm also +1 on option (2).
I'm also +1 on option (2). Colin W.
"Gary Ruben" <gazzar@email.com> writes:
In summary, I think te behaviour should be
print str(ArrayOfErr([909., 802., 677., 585.], 1.0)) [909.0 +1.0/-1.0 802.0 +1.0/-1.0 677.0 +1.0/-1.0 585.0 +1.0/-1.0 ]
print repr(ArrayOfErr([909., 802., 677., 585.], 1.0)) ObjectArray([Err(909.0,1.0,1.0), Err(802.0,1.0,1.0), Err(677.0,1.0,1.0), Err(585.0,1.0,1.0)])
Such behavior would seem sensible, but I don't think it's what the builtin collections do (not sure why):
n = 1.1 n 1.1000000000000001 [n] [1.1000000000000001] str(n) '1.1' str([n]) '[1.1000000000000001]'
'as
participants (9)
-
Alexander Schmolck -
Chris Barker -
Colin J. Williams -
David M. Cooke -
Fernando Perez -
Gary Ruben -
Perry Greenfield -
Rick White -
Todd Miller