[Matrix-SIG] Numeric Pickling
Emmanuel Viennet
emmanuel.viennet@lipn.univ-paris13.fr
Thu, 24 Jun 1999 22:28:28 +0200
The current (LNLL11) Numeric pickling mecanism
seems broken: it does not handle anymore byte-ordering
issues.
e.g., if we write data on a BigEndian machine:
>>> x = array( [ 1234567, -1234567 ] )
>>> pickle.dump( x, open( 'test.pck', 'w' ) )
and then read the file on a LittleEndian (eg a PC)
>>> x = pickle.load( open('test.pck') )
we get x == array([-2016013824, 2032791039])
Of course, a call to x.byteswapped() fixes the problem,
but pickled objects are supposed to be portables across
different architectures !
The problem is that there is currently no information in the
pickle file about byte-ordering.
I propose to change the pickling code (at the end of Numeric.py) to:
# --
import copy_reg
def array_constructor(shape, typecode, thestr, Endian=LittleEndian):
print 'Endian=%s' % Endian
x = fromstring(thestr, typecode)
x.shape = shape
if LittleEndian != Endian:
return x.byteswapped()
else:
return x
def pickle_array(a):
return array_constructor, (a.shape, a.typecode(), a.tostring(),
LittleEndian)
copy_reg.pickle(ArrayType, pickle_array, array_constructor)
# --
This solution allows to read existing files, defaulting to
current (buggy) behavior (assume native byte ordering if no
information in pickle).
Emmanuel
--
Emmanuel Viennet: <viennet@lipn.univ-paris13.fr>
LIPN - Institut Galilee - Universite Paris-Nord
93430 Villetaneuse - France
http://www-lipn.univ-paris13.fr/~viennet/