Numeric precision
Over on the scipy-dev mailing list, Travis Oliphant raised a question which is of interest to the folks over here as well. To summarize very briefly, Travis wondered whether the rule of "safest conversion" might require that an integer to float computation conversion on a 64-bit platform would require promotion of, for example, sqrt(2) to a long double (128 bits). I suggested that it would make the most sense for an M-bit integer to be converted to an N-bit real, where N>=M, on all platforms. For example, sqrt(2) would become a 32-bit real if 2 was a 32-bit integer on the plaform. I am not sure if this is a change from current Numeric and numarray practice, but wanted to give a heads-up over here. You can read the entire original thread beginning at http://www.scipy.org/mailinglists/mailman?fn=scipy-dev/2005-October/003403.h...
Hi,
The unaddressed problem is how the user will see the result. This type of
thing leads type mismatches especially when doing array indexing. In that
case it is a really bad thing to do. But in terms of mathematical
computations it is usually a given that the highest possible precision is
used when possible.
Regards
Bruce
On 10/10/05, Stephen Walton
Over on the scipy-dev mailing list, Travis Oliphant raised a question which is of interest to the folks over here as well. To summarize very briefly, Travis wondered whether the rule of "safest conversion" might require that an integer to float computation conversion on a 64-bit platform would require promotion of, for example, sqrt(2) to a long double (128 bits). I suggested that it would make the most sense for an M-bit integer to be converted to an N-bit real, where N>=M, on all platforms. For example, sqrt(2) would become a 32-bit real if 2 was a 32-bit integer on the plaform.
I am not sure if this is a change from current Numeric and numarray practice, but wanted to give a heads-up over here. You can read the entire original thread beginning at
http://www.scipy.org/mailinglists/mailman?fn=scipy-dev/2005-October/003403.h...
------------------------------------------------------- This SF.Net email is sponsored by: Power Architecture Resource Center: Free content, downloads, discussions, and more. http://solutions.newsforge.com/ibmarch.tmpl _______________________________________________ Numpy-discussion mailing list Numpy-discussion@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/numpy-discussion
participants (2)
-
Bruce Southey
-
Stephen Walton