Settable defaulting to decimal instead of float
Hi There, Settable defaulting to decimal instead of float It would be good to be able to use decimal automatically instead of float if there is a setting. For example an environment variable or a flag file. Where and when accuracy is more important than speed, the user could set this flag, and calculate with decimal numbers as learned in the school. I think several people would use this function Best regards, George
George requested this feature on the bug tracker: http://bugs.python.org/issue29223 George was asked to start a discusson on this list. I posted the following comment before closing the issue: You are not the first one to propose the idea. 2012: "make decimal the default non-integer instead of float?" https://mail.python.org/pipermail/python-ideas/2012-September/016250.html 2014: "Python Numbers as Human Concept Decimal System" https://mail.python.org/pipermail/python-ideas/2014-March/026436.html Related discussions: 2008: "Decimal literal?" https://mail.python.org/pipermail/python-ideas/2008-December/002379.html 2015: "Python Float Update" https://mail.python.org/pipermail/python-ideas/2015-June/033787.html PEP 240 "Adding a Rational Literal to Python". Victor
On 12 January 2017 at 10:28, Victor Stinner <victor.stinner@gmail.com> wrote:
OK, but without additional detail (for example, how would the proposed flag work, if the main module imports module A, then would float literals in A be decimal or binary? Both could be what the user wants) it's hard to comment. And as you say, most of this has been discussed before, so I'd like to see references back to the previous discussions in any proposal, with explanations of how the new proposal addresses the objections raised previously. Paul
I think such proposals are special cases of a general theme: a compiler pragma, similar to "from __future__", to make Python support domain-specific syntax in the current file. Whether it's decimal literals or matrix/vector literals etc. I think it will be nice to make some tool, external to Python, that will allow defining such "sibling languages" (transpiled into Python) easily and uniformly. Elazar בתאריך יום ה׳, 12 בינו' 2017, 13:21, מאת Paul Moore <p.f.moore@gmail.com>:
Something like: from __syntax__ import decimal_literal which would feed the rest of the file through the "decimal_literal" transpiler. (and not influence anything in other files). Not sure if you would want to support multiple transpilers per file. Note that Racket has something similar with their initial "#lang ..." directive. That only allows a single "language". Possibly wisely so. Stephan 2017-01-12 12:59 GMT+01:00 אלעזר <elazarg@gmail.com>:
2017-01-12 13:13 GMT+01:00 Stephan Houben <stephanh42@gmail.com>:
Something like: from __syntax__ import decimal_literal
IMHO you can already implement that with a third party library, see for example: https://github.com/lihaoyi/macropy It also reminds me my PEP 511 which would open the gate for any kind of Python preprocessor :-) https://www.python.org/dev/peps/pep-0511/ Victor
On 12 January 2017 at 15:34, Victor Stinner <victor.stinner@gmail.com> wrote:
PEP 302 (import hooks) pretty much did that years ago :-) Just write your own processor to translate a new filetype into bytecode, and register it as an import hook. There was a web framework that did that for templates not long after PEP 302 got implemented (can't recall the name any more). Paul
While I think some variation of: from __optional__ import decimal_literal Might be reasonable, I'd probably rather see something like: X = 1.1D However: (thank you Chris and Stephen) -- Decimal is NOT a panacea, nor any more "accurate" than binary floating point. It is still floating point, it is still imprecise, it still can't represent all rational numbers, even within a range. The ONLY advantage is that it gives people the warm and fuzzies because they are used to being able to represent 1/10 exactly, while not representing 1/3 exactly. But 1/10 is only special BECAUSE of Decimal representation itself. I actually think that the Decimal docs over-sell its usefulness. For instance, it really isn't more suitable for money than binary floating point if you round your outputs. Decimal does provide variable precision, which does help. With a 64 bit float, you lose cent precision around a trillion dollars. But that's a precision issue, not a binary vs Decimal issue. And a float128 would work fine for more money than I'll ever have to deal with! If you really want to do money right, you should use a money type that is exact and follows the appropriate accounting rules for rounding. Probably fixed point. (There are a couple money packages on pypi -- no idea if they are any good) In short: I'm wary of the idea that most people would be better off with Decimal. It's really a special purpose type, and I think it's better if the issues with floating point precision make themselves obvious sooner than later. -CHB Sorry for the top-post -- I hate this phone email client.... Sent from my iPhone
On Thu, Jan 12, 2017 at 6:40 PM, Stephen J. Turnbull < turnbull.stephen.fw@u.tsukuba.ac.jp> wrote:
However: (thank you Chris and Stephen) --
I think you mean "Stephan". :-)
Yes -- should have looked back at the thread! -CHB -- Christopher Barker, Ph.D. Oceanographer Emergency Response Division NOAA/NOS/OR&R (206) 526-6959 voice 7600 Sand Point Way NE (206) 526-6329 fax Seattle, WA 98115 (206) 526-6317 main reception Chris.Barker@noaa.gov
On Thu, Jan 12, 2017, at 06:20, Paul Moore wrote:
Having them be decimal is impossible and it therefore hadn't even occurred to me it might be what the user wanted. Though what might be interesting would be to have a mode or variant of the language where *the float type* is, say, decimal128. The documentation carefully avoids guaranteeing any specific representation, or even that it is binary, and the existence of float_info.radix suggests that it may not always be the case. Implementing such a thing would be difficult, of course, and making it switchable at runtime would be even harder. Though since PyFloat_FromDouble/PyFloat_AsDouble would still necessarily be part of the API, it would simply be a case of "modules that are unaware that float may use C double for its internal representation may silently lose precision" Question: Is Py_AS_DOUBLE, which directly accesses a struct field, part of the stable ABI? If so, it may be necessary for every float object to continue carrying around a C double as an 'alternate representation'. Defining it unconditionally to have both representations would also make it somewhat easier to make the behavior a runtime switch, since it would simply change which value is considered authoritative, though it would make everything unconditionally slower as every time a float is constructed the alternate representation must be calculated.
On 12 January 2017 at 09:04, George Fischhof <george@fischhof.hu> wrote:
If by "defaulting" you mean having floating point literals default to decimal, this has been discussed before and would break a lot of code, so is realistically not going to happen. If that's not what you mean, pretty much everything else can be done in your code (use "Decimal()" in place of "float()", etc). It's unlikely that there's a practical suggestion here that hasn't been discussed before and rejected, but if you have something specific to suggest, then you'll have to clarify your proposal with a lot more detail. But I wouldn't bother unless you can also demonstrate that your proposal avoids breaking backward compatibility (and "users can choose whether to use it via a flag" isn't sufficient - it doesn't help library modules for instance) as it will have no chance of being accepted unless it's backward compatible. Regards, Paul
On 12 January 2017 at 20:30, Paul Moore <p.f.moore@gmail.com> wrote:
It's unlikely that there's a practical suggestion here that hasn't been discussed before and rejected
There's one practical decimal-literal-related suggestion which hasn't been rejected yet: adding a true decimal literal based on decimal128 semantics *without* configurable context support (so compile time constant folding can work normally rather than all operations needing to be deferred to runtime). Folks that wanted to fiddle with the context settings would still need to use decimal.Decimal objects, but there'd also be a readily available builtin base10 counterpart to the binary "float" type. As far as I know the main barrier to that approach is simply the lack of folks with the time, interest, and expertise needed to implement, review, and document it, rather than it being an objectionable proposal at the language design level. (There would be some concerns around potential confusion between when to use the default binary literals and when to use the decimal literals, but those concerns arise anyway - the discrepancies between binary and decimal arithmetic are just one of those unfortunate facts of computing at this point) Cheers, Nick. -- Nick Coghlan | ncoghlan@gmail.com | Brisbane, Australia
On Thu, Jan 12, 2017 at 11:07 PM, Nick Coghlan <ncoghlan@gmail.com> wrote:
Most of the time one of my students talks to me about decimal vs binary, they're thinking that a decimal literal (or converting the default non-integer literal to be decimal) is a panacea to the "0.1 + 0.2 != 0.3" problem. Perhaps the real solution is a written-up explanation of why binary floating point is actually a good thing, and not just a backward-compatibility requirement? ChrisA
Hi Chris, 2017-01-12 13:17 GMT+01:00 Chris Angelico <rosuav@gmail.com>:
Indeed. Decimal also doesn't solve the 1/3 issue. I don't understand why people always talk about Decimal, if you want math to work "right" you probably want fractions. (With the understanding that this is for still limited value of "right".)
I have sometimes considered writing up "Why the aliens of Epsilon Eridani, whose computers use 13-valued logic, still use floating point numbers with base 2." (Short overview: analysis form first principles shows that the base should be: 1. an integral number > 1 and 2. as small as possible (to minmax the relative rounding error)) The list of candidate bases satisfying these criteria is: 2. Stephan
On Thu, Jan 12, 2017 at 11:50 PM, Stephan Houben <stephanh42@gmail.com> wrote:
My usual go-to response is that if you want perfect arithmetic with no rounding errors, your *ONLY* option is symbolic math, where sqrt(8) returns 2√2. It's pretty obvious that this gets unwieldy really fast, and rounding errors are a fact of life :) Rationals have their own problems (eg it's nearly impossible to eyeball them for size once they get big), and still don't solve everything else.
That's exactly the sort of thing I'm talking about. Among other things, only binary floating point guarantees that x <= (x+y)/2 <= y for any x <= y. (At least, I think only binary - I know decimal can't ensure that, and I haven't tested everything in between.) You're way more an expert on this than I am - my skill consists of reading what other people have written and echoing it to people :) ChrisA
On 12 January 2017 at 12:07, Nick Coghlan <ncoghlan@gmail.com> wrote:
AIUI, this would be an explicit decimal literal, something like 1.5D, rather than simply making all *existing* float literals decimal. Which isn't what I understood the OP to be asking for. Anyway, the OP needs to clarify. Paul
On 12.01.2017 10:04, George Fischhof wrote:
I don't think having this configurable is a good idea. Too much code would break as a result and become unusable for people wanting to use such functionality, so it would be of limited value. The above is similar to what we had in Python 2 for enabling Unicode literals everywhere (the -U option). It was added as experiment at the time. No one used it due to the massive breakage it caused in the stdlib. Why not explicitly code for your use case ? -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Experts (#1, Jan 12 2017)
::: We implement business ideas - efficiently in both time and costs ::: eGenix.com Software, Skills and Services GmbH Pastor-Loeh-Str.48 D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg Registered at Amtsgericht Duesseldorf: HRB 46611 http://www.egenix.com/company/contact/ http://www.malemburg.com/
George requested this feature on the bug tracker: http://bugs.python.org/issue29223 George was asked to start a discusson on this list. I posted the following comment before closing the issue: You are not the first one to propose the idea. 2012: "make decimal the default non-integer instead of float?" https://mail.python.org/pipermail/python-ideas/2012-September/016250.html 2014: "Python Numbers as Human Concept Decimal System" https://mail.python.org/pipermail/python-ideas/2014-March/026436.html Related discussions: 2008: "Decimal literal?" https://mail.python.org/pipermail/python-ideas/2008-December/002379.html 2015: "Python Float Update" https://mail.python.org/pipermail/python-ideas/2015-June/033787.html PEP 240 "Adding a Rational Literal to Python". Victor
On 12 January 2017 at 10:28, Victor Stinner <victor.stinner@gmail.com> wrote:
OK, but without additional detail (for example, how would the proposed flag work, if the main module imports module A, then would float literals in A be decimal or binary? Both could be what the user wants) it's hard to comment. And as you say, most of this has been discussed before, so I'd like to see references back to the previous discussions in any proposal, with explanations of how the new proposal addresses the objections raised previously. Paul
I think such proposals are special cases of a general theme: a compiler pragma, similar to "from __future__", to make Python support domain-specific syntax in the current file. Whether it's decimal literals or matrix/vector literals etc. I think it will be nice to make some tool, external to Python, that will allow defining such "sibling languages" (transpiled into Python) easily and uniformly. Elazar בתאריך יום ה׳, 12 בינו' 2017, 13:21, מאת Paul Moore <p.f.moore@gmail.com>:
Something like: from __syntax__ import decimal_literal which would feed the rest of the file through the "decimal_literal" transpiler. (and not influence anything in other files). Not sure if you would want to support multiple transpilers per file. Note that Racket has something similar with their initial "#lang ..." directive. That only allows a single "language". Possibly wisely so. Stephan 2017-01-12 12:59 GMT+01:00 אלעזר <elazarg@gmail.com>:
2017-01-12 13:13 GMT+01:00 Stephan Houben <stephanh42@gmail.com>:
Something like: from __syntax__ import decimal_literal
IMHO you can already implement that with a third party library, see for example: https://github.com/lihaoyi/macropy It also reminds me my PEP 511 which would open the gate for any kind of Python preprocessor :-) https://www.python.org/dev/peps/pep-0511/ Victor
On 12 January 2017 at 15:34, Victor Stinner <victor.stinner@gmail.com> wrote:
PEP 302 (import hooks) pretty much did that years ago :-) Just write your own processor to translate a new filetype into bytecode, and register it as an import hook. There was a web framework that did that for templates not long after PEP 302 got implemented (can't recall the name any more). Paul
While I think some variation of: from __optional__ import decimal_literal Might be reasonable, I'd probably rather see something like: X = 1.1D However: (thank you Chris and Stephen) -- Decimal is NOT a panacea, nor any more "accurate" than binary floating point. It is still floating point, it is still imprecise, it still can't represent all rational numbers, even within a range. The ONLY advantage is that it gives people the warm and fuzzies because they are used to being able to represent 1/10 exactly, while not representing 1/3 exactly. But 1/10 is only special BECAUSE of Decimal representation itself. I actually think that the Decimal docs over-sell its usefulness. For instance, it really isn't more suitable for money than binary floating point if you round your outputs. Decimal does provide variable precision, which does help. With a 64 bit float, you lose cent precision around a trillion dollars. But that's a precision issue, not a binary vs Decimal issue. And a float128 would work fine for more money than I'll ever have to deal with! If you really want to do money right, you should use a money type that is exact and follows the appropriate accounting rules for rounding. Probably fixed point. (There are a couple money packages on pypi -- no idea if they are any good) In short: I'm wary of the idea that most people would be better off with Decimal. It's really a special purpose type, and I think it's better if the issues with floating point precision make themselves obvious sooner than later. -CHB Sorry for the top-post -- I hate this phone email client.... Sent from my iPhone
On Thu, Jan 12, 2017 at 6:40 PM, Stephen J. Turnbull < turnbull.stephen.fw@u.tsukuba.ac.jp> wrote:
However: (thank you Chris and Stephen) --
I think you mean "Stephan". :-)
Yes -- should have looked back at the thread! -CHB -- Christopher Barker, Ph.D. Oceanographer Emergency Response Division NOAA/NOS/OR&R (206) 526-6959 voice 7600 Sand Point Way NE (206) 526-6329 fax Seattle, WA 98115 (206) 526-6317 main reception Chris.Barker@noaa.gov
On Thu, Jan 12, 2017, at 06:20, Paul Moore wrote:
Having them be decimal is impossible and it therefore hadn't even occurred to me it might be what the user wanted. Though what might be interesting would be to have a mode or variant of the language where *the float type* is, say, decimal128. The documentation carefully avoids guaranteeing any specific representation, or even that it is binary, and the existence of float_info.radix suggests that it may not always be the case. Implementing such a thing would be difficult, of course, and making it switchable at runtime would be even harder. Though since PyFloat_FromDouble/PyFloat_AsDouble would still necessarily be part of the API, it would simply be a case of "modules that are unaware that float may use C double for its internal representation may silently lose precision" Question: Is Py_AS_DOUBLE, which directly accesses a struct field, part of the stable ABI? If so, it may be necessary for every float object to continue carrying around a C double as an 'alternate representation'. Defining it unconditionally to have both representations would also make it somewhat easier to make the behavior a runtime switch, since it would simply change which value is considered authoritative, though it would make everything unconditionally slower as every time a float is constructed the alternate representation must be calculated.
On 12 January 2017 at 09:04, George Fischhof <george@fischhof.hu> wrote:
If by "defaulting" you mean having floating point literals default to decimal, this has been discussed before and would break a lot of code, so is realistically not going to happen. If that's not what you mean, pretty much everything else can be done in your code (use "Decimal()" in place of "float()", etc). It's unlikely that there's a practical suggestion here that hasn't been discussed before and rejected, but if you have something specific to suggest, then you'll have to clarify your proposal with a lot more detail. But I wouldn't bother unless you can also demonstrate that your proposal avoids breaking backward compatibility (and "users can choose whether to use it via a flag" isn't sufficient - it doesn't help library modules for instance) as it will have no chance of being accepted unless it's backward compatible. Regards, Paul
On 12 January 2017 at 20:30, Paul Moore <p.f.moore@gmail.com> wrote:
It's unlikely that there's a practical suggestion here that hasn't been discussed before and rejected
There's one practical decimal-literal-related suggestion which hasn't been rejected yet: adding a true decimal literal based on decimal128 semantics *without* configurable context support (so compile time constant folding can work normally rather than all operations needing to be deferred to runtime). Folks that wanted to fiddle with the context settings would still need to use decimal.Decimal objects, but there'd also be a readily available builtin base10 counterpart to the binary "float" type. As far as I know the main barrier to that approach is simply the lack of folks with the time, interest, and expertise needed to implement, review, and document it, rather than it being an objectionable proposal at the language design level. (There would be some concerns around potential confusion between when to use the default binary literals and when to use the decimal literals, but those concerns arise anyway - the discrepancies between binary and decimal arithmetic are just one of those unfortunate facts of computing at this point) Cheers, Nick. -- Nick Coghlan | ncoghlan@gmail.com | Brisbane, Australia
On Thu, Jan 12, 2017 at 11:07 PM, Nick Coghlan <ncoghlan@gmail.com> wrote:
Most of the time one of my students talks to me about decimal vs binary, they're thinking that a decimal literal (or converting the default non-integer literal to be decimal) is a panacea to the "0.1 + 0.2 != 0.3" problem. Perhaps the real solution is a written-up explanation of why binary floating point is actually a good thing, and not just a backward-compatibility requirement? ChrisA
Hi Chris, 2017-01-12 13:17 GMT+01:00 Chris Angelico <rosuav@gmail.com>:
Indeed. Decimal also doesn't solve the 1/3 issue. I don't understand why people always talk about Decimal, if you want math to work "right" you probably want fractions. (With the understanding that this is for still limited value of "right".)
I have sometimes considered writing up "Why the aliens of Epsilon Eridani, whose computers use 13-valued logic, still use floating point numbers with base 2." (Short overview: analysis form first principles shows that the base should be: 1. an integral number > 1 and 2. as small as possible (to minmax the relative rounding error)) The list of candidate bases satisfying these criteria is: 2. Stephan
On Thu, Jan 12, 2017 at 11:50 PM, Stephan Houben <stephanh42@gmail.com> wrote:
My usual go-to response is that if you want perfect arithmetic with no rounding errors, your *ONLY* option is symbolic math, where sqrt(8) returns 2√2. It's pretty obvious that this gets unwieldy really fast, and rounding errors are a fact of life :) Rationals have their own problems (eg it's nearly impossible to eyeball them for size once they get big), and still don't solve everything else.
That's exactly the sort of thing I'm talking about. Among other things, only binary floating point guarantees that x <= (x+y)/2 <= y for any x <= y. (At least, I think only binary - I know decimal can't ensure that, and I haven't tested everything in between.) You're way more an expert on this than I am - my skill consists of reading what other people have written and echoing it to people :) ChrisA
On 12 January 2017 at 12:07, Nick Coghlan <ncoghlan@gmail.com> wrote:
AIUI, this would be an explicit decimal literal, something like 1.5D, rather than simply making all *existing* float literals decimal. Which isn't what I understood the OP to be asking for. Anyway, the OP needs to clarify. Paul
On 12.01.2017 10:04, George Fischhof wrote:
I don't think having this configurable is a good idea. Too much code would break as a result and become unusable for people wanting to use such functionality, so it would be of limited value. The above is similar to what we had in Python 2 for enabling Unicode literals everywhere (the -U option). It was added as experiment at the time. No one used it due to the massive breakage it caused in the stdlib. Why not explicitly code for your use case ? -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Experts (#1, Jan 12 2017)
::: We implement business ideas - efficiently in both time and costs ::: eGenix.com Software, Skills and Services GmbH Pastor-Loeh-Str.48 D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg Registered at Amtsgericht Duesseldorf: HRB 46611 http://www.egenix.com/company/contact/ http://www.malemburg.com/
participants (12)
-
Chris Angelico -
Chris Barker -
Chris Barker - NOAA Federal
-
George Fischhof -
M.-A. Lemburg -
Nick Coghlan -
Paul Moore -
Random832 -
Stephan Houben -
Stephen J. Turnbull -
Victor Stinner -
אלעזר