[Python-Dev] Re: marshal / unmarshal

[Michael Hudson]

Tim Peters <tim.peters at gmail.com> writes:

> marshal shouldn't be representing doubles as decimal strings to begin
> with.  All code for (de)serialing C doubles should go thru
> _PyFloat_Pack8() and _PyFloat_Unpack8().  cPickle (proto >= 1) and
> struct (std mode) already do; marshal is the oddball.
>
> But as the docs (floatobject.h) for these say:
>
> ...
>  * Bug:  What this does is undefined if x is a NaN or infinity.
>  * Bug:  -0.0 and +0.0 produce the same string.
>  */
> PyAPI_FUNC(int) _PyFloat_Pack4(double x, unsigned char *p, int le);
> PyAPI_FUNC(int) _PyFloat_Pack8(double x, unsigned char *p, int le);
> ...
>  * Bug:  What this does is undefined if the string represents a NaN or
>  * infinity.
>  */
> PyAPI_FUNC(double) _PyFloat_Unpack4(const unsigned char *p, int le);
> PyAPI_FUNC(double) _PyFloat_Unpack8(const unsigned char *p, int le);

OTOH, the implementation has this comment:

/*----------------------------------------------------------------------------
 * _PyFloat_{Pack,Unpack}{4,8}.  See floatobject.h.
 *
 * TODO:  On platforms that use the standard IEEE-754 single and double
 * formats natively, these routines could simply copy the bytes.
 */

Doing that would fix these problems, surely?[1]

The question, of course, is how to tell.  I suppose one could jsut do
it unconditionally and wait for one of the three remaining VAX
users[2] to compile Python 2.5 and then notice.

More conservatively, one could just do this on Windows, linux/most
architectures and Mac OS X.

Cheers,
mwh

[1] I'm slighyly worried about oddball systems that do insane things
    with the FPU by default -- but don't think the mooted change would
    make things any worse.

[2] Exaggeration, I realize -- but how many non 754 systems are out
    there?  How many will see Python 2.5?

-- 
  If you give someone Fortran, he has Fortran.
  If you give someone Lisp, he has any language he pleases.
    -- Guy L. Steele Jr, quoted by David Rush in comp.lang.scheme.scsh