[New-bugs-announce] [issue26229] Make number serialization ES6/V8 compatible

[Anders Rundgren]

New submission from Anders Rundgren:

ECMA has in their latest release defined that JSON elements must be ordered during serialization.  This is easy to accomplish using Python's OrderedDict.  What is less trivial is that numbers have to be formatted in a certain way as well.  I have tested 100 millions specific and random values and found out that Python 3.5.1 is mathematically identical to ES6/V8 but has some differences in formatting:

   IEEE Double        ECMAScript 6/V8            Python 3.5.1

c43211ede4974a35, -333333333333333300000,    -3.333333333333333e+20
c3fce97ca0f21056, -33333333333333336000,     -3.3333333333333336e+19
c3c7213080c1a6ac, -3333333333333334000,      -3.333333333333334e+18
c39280f39a348556, -333333333333333400,       -3.333333333333334e+17
c35d9b1f5d20d557, -33333333333333340,        -3.333333333333334e+16

c327af4c4a80aaac, -3333333333333334,         -3333333333333334.0

bf0179ec9cbd821e, -0.000033333333333333335,  -3.3333333333333335e-05
becbf647612f3696, -0.0000033333333333333333, -3.3333333333333333e-06

4024000000000000, 10,                        10.0
0000000000000000, 0,                         0.0
4014000000000000, 5,                         5.0
3f0a36e2eb1c432d, 0.00005,                   5e-05
3ed4f8b588e368f1, 0.000005,                  5e-06

3ea0c6f7a0b5ed8d, 5e-7,                      5e-07

Why could this be important?

https://github.com/Microsoft/ChakraCore/issues/149

# Python test program
import binascii
import struct
import json

f = open('c:\\es6\\numbers\\es6testfile100m.txt','rb')

l = 0;
string = '';

while True:
  byte = f.read(1);
  if len(byte) == 0:
    exit(0)
  if byte == b'\n':
    l = l + 1;
    i = string.find(',')
    if i <= 0 or i >= len(string) - 1:
      print('Bad string: ' + str(i))
      exit(0)
    hex = string[:i]
    while len(hex) < 16:
      hex = '0' + hex
    o = dict()
    o['double'] = struct.unpack('>d',binascii.a2b_hex(hex))[0]
    py3Double = json.dumps(o)[11:-1]
    es6Double = string[i + 1:]
    if es6Double != py3Double:
      es6Dpos = es6Double.find('.')
      py3Dpos = py3Double.find('.')
      es6Epos = es6Double.find('e')
      py3Epos = py3Double.find('e')
      if py3Epos > 0:
        py3Exp = int(py3Double[py3Epos + 1:])
      if es6Dpos < 0 and py3Dpos > 0:
        if es6Epos < 0 and py3Epos > 0:
          py3New = py3Double[:py3Dpos] + py3Double[py3Dpos + 1:py3Epos - len(py3Double)]
          q = py3Exp - py3Epos + py3Dpos
          while q >= 0:
            py3New += '0'
            q -= 1
          if py3New != es6Double:
            print('E1: ' + py3New)
            exit(0)
        elif py3Epos < 0:
          py3New = py3Double[:-2]
          if py3New != es6Double:
            print('E2: ' + py3New)
            exit(0)
        else:
          print (error + hex + '#' + es6Double + '#' + py3Double)
          exit(0)
      elif es6Dpos > 0 and py3Dpos > 0 and py3Epos > 0 and es6Epos < 0:
        py3New = py3Double[py3Dpos - 1:py3Dpos] + py3Double[py3Dpos + 1:py3Epos - len(py3Double)]
        q = py3Exp + 1
        while q < 0:
          q += 1
          py3New = '0' + py3New
        py3New = py3Double[0:py3Dpos - 1] + '0.' + py3New 
        if py3New != es6Double:
          print('E3: ' + py3New + '#' + es6Double)
          exit(0)
      elif es6Dpos == py3Dpos and py3Epos > 0 and es6Epos > 0:
        py3New = py3Double[:py3Epos + 2] + str(abs(py3Exp))
        if py3New != es6Double:
          print('E4: ' + py3New + '#' + es6Double)
          exit(0)
      elif es6Dpos > 0 and py3Dpos < 0 and py3Epos > 0 and es6Epos < 0:
        py3New = py3Double[:py3Epos - len(py3Double)]
        q = py3Exp + 1
        while q < 0:
          q += 1
          py3New = '0' + py3New
        py3New = '0.' + py3New 
        if py3New != es6Double:
          print('E5: ' + py3New + '#' + es6Double)
          exit(0)
      else:
        print ('Unexpected: ' + hex + '#' + es6Double + '#' + py3Double)
        exit(0)
    string = ''
    if l % 10000 == 0:
      print(l)
  else:
    string += byte.decode(encoding='UTF-8')

----------
components: Interpreter Core
messages: 259105
nosy: anders.rundgren.net at gmail.com
priority: normal
severity: normal
status: open
title: Make number serialization ES6/V8 compatible
type: enhancement
versions: Python 3.5

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue26229>
_______________________________________