Converting IBM Floats..Help..

[Anton Vredegoor]

"Ian Sparks" <Ian.Sparks at etrials.com> wrote:

>I'm trying to read from a file which contains numbers encoded in IBM360 (Big-Endian) Floating point format. The IBM stores its numbers in the following format :
>
>*
>* IBM format:
>* 6       5                 0
>* 3       1                 0
>*
>* SEEEEEEEMMMM ......... MMMM
>*
>* Sign bit, 7 bit exponent, 56 bit fraction. Exponent is
>* excess 64. The fraction is multiplied by a power of 16 of
>* the actual exponent. Normalized floating point numbers are
>* represented with the radix point immediately to the left of
>* the high order hex fraction digit.
>
>I know what's in this file so :
>
> data = list(file.read(8))
> print data
> 
> 155 = ['B', '\x9b', '\x00', '\x00', '\x00', '\x00', '\x00', '\x00'] 
>  77 = ['B', 'M', '\x00', '\x00', '\x00', '\x00', '\x00', '\x00']
>   1 = ['A', '\x10', '\x00', '\x00', '\x00', '\x00', '\x00', '\x00']
>   0 = ['\x00', '\x00', '\x00', '\x00', '\x00', '\x00', '\x00', '\x00']

Some time ago there was a question on this group about 4-bytes IBM
floats, and maybe questions about other formats will follow in the
future. Here's a flexible -but possibly slow?- approach to handle
these kind of problems. It uses a stream of bytes (a file for example)
as input and returns a series of floats.

from itertools import islice

def _bits(i):
    return [('01'[i>>j & 1]) for j in range(8)][::-1] 

_table = dict([(chr(i),_bits(i)) for i in range(256)])

def bitstream(bytes):
    for byte in bytes:
        for bit in _table[byte]:
            yield bit
            
def groupby(gen,sizes):
    while 1:
        for size in sizes:
            yield islice(gen,size)

def tolongs(gen):
    def tolong(x): return long(''.join(x),2)
    for g in gen:
        try: yield map(tolong,g)
        except ValueError: break

def decode(gen):
    for sign, exponent, mantissa in gen:        
        sign = [1,-1][sign] 
        exponent = 16**(exponent & 0x7f -64)
        mantissa = mantissa/float(16**14)
        yield sign*exponent*mantissa
    
def float_decode(bytes):
    bits = bitstream(bytes)       # turn it into a stream of bits
    gb_1 = groupby(bits,[1,7,56]) # make groups of size 1,7 and 56
    gb_2 = groupby(gb_1,[3])      # group again, take 3 at a time
    gl = tolongs(gb_2)            # turn it into groups of three longs
    for f in decode(gl):          # turn 3 longs into a float
        yield f
        
def test():
    bytes = ('B\x9b\x00\x00\x00\x00\x00\x00'
        'BM\x00\x00\x00\x00\x00\x00'
        'A\x10\x00\x00\x00\x00\x00\x00'
        '\x00\x00\x00\x00\x00\x00\x00\x00')
    for f in float_decode(bytes):
        print f

if __name__ == '__main__': 
    test()

output:

155.0
77.0
1.0
0.0

Anton