[ python-Bugs-1084279 ] status of small floats in xml-rpc ?

Fri Dec 17 21:30:10 CET 2004

Bugs item #1084279, was opened at 2004-12-13 05:07
Message generated for change (Comment added) made by tim_one
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1084279&group_id=5470

Category: Python Library
Group: Python 2.4
Status: Open
Resolution: None
Priority: 5
Submitted By: Antoine Pitrou (pitrou)
Assigned to: Nobody/Anonymous (nobody)
Summary: status of small floats in xml-rpc ?

Initial Comment:
Hi,

I've been reading through the xmlrpclib.py code to see
that floats are marshalled the following way:

    def dump_double(self, value, write):
        write("<value><double>")
        write(repr(value))
        write("</double></value>\n")
    dispatch[FloatType] = dump_double

Using repr() means that small or big floats, for
example, will be output using the exponent notation:
>>> repr(2**-15)
'3.0517578125e-05'

Unfortunately, the XML-RPC specification does not allow
this notation:
"At this time, only decimal point notation is allowed,
a plus or a minus, followed by any number of numeric
characters, followed by a period and any number of
numeric characters. Whitespace is not allowed. The
range of allowable values is implementation-dependent,
is not specified."
(http://www.xmlrpc.com/spec)

Thus floats marshalled using xmlrpclib may not be
readable using other XML-RPC implementations.

(I don't have any concrete data about this though)

----------------------------------------------------------------------

>Comment By: Tim Peters (tim_one)
Date: 2004-12-17 15:30

Message:
Logged In: YES 
user_id=31435

FWIW, I'd leave this alone.  The XML-RPC spec is braindead 
when it comes to floats, and any implementation that didn't 
accept scientific notation would have to go out of its way to 
cripple the facilities all languages with a string->double 
function supply.

Raymond, under any IEEE-conforming string->double 
implementation, eval(repr(x))==x will hold provided that repr
(x) be roughly correct to at least 17 significant decimal digits, 
and that x is finite.  It doesn't matter whether the string uses 
exponent notation.  OTOH, IIRC Python got in trouble once 
with some old Solaris string->double C routine that went 
insane if the string had "too many" characters.  Indeed, 
making bad assumptions about maximum string length seems 
a lot more likely to me than that someone has a string-
>double routine that can't deal with exponents.

----------------------------------------------------------------------

Comment By: Raymond Hettinger (rhettinger)
Date: 2004-12-16 05:32

Message:
Logged In: YES 
user_id=80475

After more thought, I'm concerned that switching to full
decimal notation will break the guarantee eval(repr(x))==x.
  Also, Skip's thoughts seem reasonable. Rather than
switching to a non-compact format, it may be best to way for
the spec to catchup with real implementations.

----------------------------------------------------------------------

Comment By: Raymond Hettinger (rhettinger)
Date: 2004-12-13 17:19

Message:
Logged In: YES 
user_id=80475

Various slightly unsatisfactory answers:
* It started out that way because that's what C did.
* It stayed that way because no one has requested it
* It may be that outside of XMLRPC there are very few valid
use cases.
* Future needs can be met by the decimal module.

----------------------------------------------------------------------

Comment By: Antoine Pitrou (pitrou)
Date: 2004-12-13 14:04

Message:
Logged In: YES 
user_id=133955

I'm gonna ask a stupid question (I'm quite a beginner in
Python). Why isn't there a %f-life formatting code for doing
just what you wrote - printing the float in full precision
in non-exponent format ?

----------------------------------------------------------------------

Comment By: Raymond Hettinger (rhettinger)
Date: 2004-12-13 13:12

Message:
Logged In: YES 
user_id=80475

The thought was "be liberal in what you accept and be strict
on was is emitted."  Still, the question is valid.

For C, it looks like strtod() is at the root of everything
converting from floats.  The spec lays no limits on the
input format (exponents vs full decimal representation). 
Instead, it just checks that the result is inside the range
of representable values and did not underflow.  

Some experiments with MSC6 confirm that the full range may
be entered as regular decimals.  Experiments with Perl show
the same result.  I suspect all TCL and Ruby also have
strtod() at the core.

----------------------------------------------------------------------

Comment By: Skip Montanaro (montanaro)
Date: 2004-12-13 10:38

Message:
Logged In: YES 
user_id=44345

I understand what you''re doing, but I wonder if in the
process interoperability with other XML-RPC implementations
might actually get worse.  While the spec doesn't support
exponential notation, most programming languages do, and
probably happily accept "1.234e-147" as a floating point
input.  Python seems not to have problems accepting floating
point inputs with huge numbers of zeroes before or after the
decimal point, but other languages may not be quite so
forgiving.  I think we need to be a bit careful before
changing our implementation.  At the very least we should
probably verify the acceptance of non_e_repr-generated
strings by a few other languages commonly used with XML-RPC
(C, Perl, Tcl, Ruby?).

----------------------------------------------------------------------

Comment By: Raymond Hettinger (rhettinger)
Date: 2004-12-13 08:13

Message:
Logged In: YES 
user_id=80475

See attached patch.

----------------------------------------------------------------------

Comment By: Raymond Hettinger (rhettinger)
Date: 2004-12-13 07:40

Message:
Logged In: YES 
user_id=80475

I loaded a recipe for a helper function that meets the spec:

http://aspn.activestate.com/ASPN/Cookbook/Python/Recipe/358361

Inserting and applying the helper function ought to be
backwards compatible (the new encoding is readable by
previous versions).

Martin, do you agree with approach and concur that it should
be backported?

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1084279&group_id=5470