[Matrix-SIG] Numeric Pickling

Emmanuel Viennet emmanuel.viennet@lipn.univ-paris13.fr
Thu, 24 Jun 1999 22:28:28 +0200


The current (LNLL11) Numeric pickling mecanism
seems broken: it does not handle anymore byte-ordering 
issues.

e.g., if we write data on a BigEndian machine:
>>> x = array( [ 1234567, -1234567 ] )
>>> pickle.dump( x, open( 'test.pck', 'w' ) )

and then read the file on a LittleEndian (eg a PC)

>>> x = pickle.load( open('test.pck') )

we get x == array([-2016013824,  2032791039])

Of course, a call to x.byteswapped() fixes the problem,
but pickled objects are supposed to be portables across 
different architectures !
The problem is that there is currently no information in the
pickle file about byte-ordering. 

I propose to change the pickling code (at the end of Numeric.py) to:

# --
import copy_reg

def array_constructor(shape, typecode, thestr, Endian=LittleEndian):
    print 'Endian=%s' % Endian
    x = fromstring(thestr, typecode)
    x.shape = shape
    if LittleEndian != Endian:
	return x.byteswapped()
    else:
	return x

def pickle_array(a):
    return array_constructor, (a.shape, a.typecode(), a.tostring(),
LittleEndian)  

copy_reg.pickle(ArrayType, pickle_array, array_constructor)
# -- 

This solution allows to read existing files, defaulting to
current (buggy) behavior (assume native byte ordering if no
information in pickle).


Emmanuel

-- 
Emmanuel Viennet: <viennet@lipn.univ-paris13.fr>
LIPN - Institut Galilee - Universite Paris-Nord       
93430 Villetaneuse -  France
http://www-lipn.univ-paris13.fr/~viennet/