Here's an email Todd Miller sent me (I hoped he'd send it directly to the list, but I'll forward it. Todd, I hope you don't mind.) Todd Miller wrote:
On Fri, 2005-04-08 at 15:46 -0700, Andrew Straw wrote:
Hi Todd,
Could you join in on this thread? I think you wrote the ieeespecial stuff in numarray, so it's clear you have a much better understanding of the issues than I do...
Cheers! Andrew
My own understanding is limited, but I can say a few things that might make the status of numarray clearer. My assumptions for numarray were that:
1. Floating point values are 32-bit or 64-bit entities which are stored in IEEE-754 format. This is a basic assumption of numarray.ieeespecial so I expect it simply won't work on a VAX. There's no checking for this.
2. The platforms that I care about, AMD/Intel Windows/Linux, PowerPC OS-X, and Ultra-SPARC Solaris, all seem to provide IEEE-754 floating point. ieeespecial has been tested to work there.
3. I viewed IEEE-754 floating point numbers as 32-bit or 64-bit unsigned integers, and contiguous ranges on those integers are used to represent special values like NAN and INF. Platform byte ordering for the IEEE-754 floating point numbers mirrors byte ordering for integers so the ieeespecial NAN detection code works in a cross platform way *and* values exported from one IEEE-754 platform will work with ieeespecial when imported on another. It's important to note that special values are not unique: there is no single NAN value; it's a bit range.
4. numarray leaks IEEE-754 special values out into Python floating point scalars. This may be bad form. I do this because (1) they repr understandably if not in a platform independent way and (2) people need to get at them. I noticed recently that ieeespecial.nan == ieeespecial.nan returns incorrect answers (True!) for Python-2.3 and correct ones (False) for Python-2.4. I haven't looked at what the array version does yet: array(nan) == array(nan). The point to be taken from this is that the level at which numarray ieee special value handling works or doesn't work is really restricted to (1) detecting certain ieee-754 bit ranges (2) the basic behavior of C code for C89 complilers for array code (no guarantees) (3) the behavior of Python itself (improving).
In the context of the array protocol (looking very nice by the way) my thinking is that non-IEEE-754 floating point could be described with bit fields and that the current type codes should mean IEEE-754.
Some minor things I noticed in the array interface:
1. The packing order of bit fields is not clear. In C, my experience is that some compilers pack bit structs towards the higher order bits of an integer, and some towards the lower. More info to clarify that would be helpful.
2. I saw no mention that we're talking about a protocol. I'm sure that's clear to everyone following this discussion closely, but I didn't see it in the spec. It might make sense to allude to the C helper functions and potential for additions to the Python type struct even if they're not spelled out.
Regards, Todd
On Apr 9, 2005, at 9:54 AM, Travis Oliphant wrote:
konrad.hinsen@laposte.net wrote:
On 09.04.2005, at 01:04, Scott Gilbert wrote:
I think something we've been assuming is that the array data is basically IEEE-754 compliant (maybe it needs to be byteswapped). If that's not true, then we're going to need some new typecodes. We're not supporting the ability to pass VAX floating point around (Are we????).
No, in moving from the struct modules character codes we are trying to do something more platform independent because it is very likely that different platforms will want to exchange binary data. IEEE-754 is a great standard to build an interface around. Data sharing was the whole reason the standard emerged and a lot of companies got on board.
This discussion has been coming up regularly for a few years. Until now the concensus has always been that Python should make no assumptions that go beyond what a C compiler can promise. Which means no assumptions about floating-point representation.
Of course the computing world is changing, and IEEE format may well be ubiquitous by now. Vaxes must be in the museum by now. But how about mainframes? IBM mainframes didn't use IEEE when I used them (last time 15 years ago), and they are still around, possibly compatible to their ancestors.
I found the following piece, written about 6 years ago interesting:
http://www.research.ibm.com/journal/rd/435/schwarz.html
Basically, it states that chips in newer IBM mainframes support the IEEE 754 standard.
Another detail to consider is that although most machines use the IEEE representation, hardly any respects the IEEE rules for floating point operations in all detail. In particular, trusting that Inf and NaN will be treated as IEEE postulates is a risky business.
But, this can be handled with platform-dependendent C-code when and if problems arise. -Travis
------------------------------------------------------- SF email is sponsored by - The IT Product Guide Read honest & candid reviews on hundreds of IT Products from real users. Discover which products truly live up to the hype. Start reading now. http://ads.osdn.com/?ad_id=6595&alloc_id=14396&op=click _______________________________________________ Numpy-discussion mailing list Numpy-discussion@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/numpy-discussion