Floating point "g" format not stripping trailing zeros
Dave Angel
davea at davea.name
Fri Feb 13 16:15:08 EST 2015
On 02/13/2015 03:33 PM, Grant Edwards wrote:
> On 2015-02-13, Ian Kelly <ian.g.kelly at gmail.com> wrote:
>> Significant digits are within the precision of the calculation.
>> Writing 1.230 indicates that the fourth digit is known to be zero.
>> Writing 1.23 outside a context of exact calculation indicates that the
>> fourth digit is unknown due to insufficient precision.
>
> I knew that, but I was asking in the context of float/decimal's
> formatting function. I didn't realize that float and/or decimal had a
> "significant digit" property, and therefore possess significant vs.
> insignificant trailing zeros when represented in base-10.
>
Just jumping in here, and somebody please correct me if I mess it up.
Built-in binary floating point (float) has a fixed number of bits for
mantissa, and separate bits for exponent and sign. Because of those
fixed number of bits, no assumption can be made as to how many of them
are relevant.
On the other hand, the Decimal package has a way that the programmer can
tell how many digits to use at each stage of the calculation. So if the
programmer bothered to set it to the correct precision, the print logic
(could) use that to decide about trailing zeroes. I have no idea
whether it does, but this thread would seem to say it does.
I also have no definite opinion as to whether that's reasonable, or
whether most calculations are done by setting digits to about twice
what's needed, and rounding at the end. I know I did exactly that when
I wrote a variable length math package to double-check the accuracy of a
fixed precision package I was developing, 40 years ago (long before the
IEEE-754 standard began meetings). The fixed precision package was
fast, and used lots of clever(?) shortcuts for speed. The variable one
was written very brute force, ran maybe 100 times slower (on another
machine), but the results could be compared with automatic algorithms.
For simple arithmetic, not too big a deal, but for transcendentals, the
error analysis was very important. For example, the fast algorithm was
a custom chebyshev, while the reference implementation was Taylor series.
--
DaveA
More information about the Python-list
mailing list