5 Mar
2014
5 Mar
'14

3:42 a.m.

Greetings,

The purpose of this idea is to expand a bit on the decimal default idea
which I submitted previously. In this idea I want to suggest the human
idea of a *python number.* The concept is very simple, yet may have far
reaching implications not only for future python(s) but also for the wider
adaptation of python in the greater academic and professional communities.

The idea of *python number* means that there are no types, no limits, no
constraints, and that all *python numbers *are dynamic. The upper level
syntax is also very simple; all* python numbers *are simply human.

My influence for this preference is rooted in the Rexx programming
language; attributed to Mike Cowlishaw, former IBM fellow. The Rexx
programming language is dynamic, and has no types. To put it more
succinctly for those of you who have not used Rexx, the only data type
is a string of characters (that's it). *Rexx numbers* are simply those
strings of characters that may be interpreted as a valid *Rexx number.*

http://books.google.com/books?id=cNiVqFmPs8AC&pg=PA100&lpg=PA100&...

The Python language might be changed to adopt the *python number*
concept for *all math processing*, unless explicitly modified. This goes
somewhat beyond using decimal floating point as a default numerical
type. It means using human numeric expressions that meet human
expectation for numeric processing by default.

Under the covers (whatever we mean by that) processing is handled
by decimal.Decimal, unless explicitly modified. What does this mean for
python users in general? Well, no more worrying about types at all... no
ints,
no floats, no rationals, no irrationals, no fractions, and certainly no
binaries.
In short, if its a human number, its a *python number.*

I am expecting that (just as in Rexx numbers) defining very clearly what
is a *python number* will be key for wide adaptation of the concept. But
there
should be no surprises for users, particularly average users. Anyone with
a middle school expectation of a numeric format should be able to use
*python numbers *without surprises. However, for advanced users the full
interface should be available (as is the context for Decimal) through coding
based on knowledge and experience, yet the default context for Decimal
should be based on average users in most environments and use cases.

It is my belief that Python should lead the way in the twenty-first century for advanced computation for academic, professional, business, and scientific communities. There is a 40 year momentum for embedded binary floats & doubles, also numeric types generally, but it is time to move forward. The technology is ready, and the need is great. Let's do it.

{shameless plug} pdeclib https://pypi.python.org/pypi/pdeclib

Thank you for your consideration. Good evening.

Mark H Harris marcus

5 Mar
5 Mar

6:57 a.m.

On Wed, Mar 5, 2014 at 2:42 PM, Mark H. Harris harrismh777@gmail.com wrote:

The idea of python number means that there are no types, no limits, no constraints, and that all python numbers are dynamic. The upper level syntax is also very simple; all python numbers are simply human.

My influence for this preference is rooted in the Rexx programming language; attributed to Mike Cowlishaw, former IBM fellow. The Rexx programming language is dynamic, and has no types. To put it more succinctly for those of you who have not used Rexx, the only data type is a string of characters (that's it). Rexx numbers are simply those strings of characters that may be interpreted as a valid Rexx number.

I've used REXX extensively. (It's the native scripting language of OS/2, which I first met in the early 1990s and am still using - albeit usually in a VM under Linux these days - as it's still a good thing, just a little left-behind.) There are two huge downsides to that kind of proposal.

1) Performance. Huge huge performance hit. I can take Pike to OS/2, an unoptimized build on an unsupported platform, and absolutely cream a REXX program at any sort of computational work. By specifically working with integers rather than "numbers", I can run code blazingly fast.

2) Accuracy. With REXX, all arithmetic is governed by a single setting of NUMERIC DIGITS, which specifies how many digits of accuracy you want to preserve. (Python can improve granularity with separate contexts, but the same issue will still apply - contexts just let you use different NUMERIC DIGITS settings simultaneously in one program.) Set it too high and performance suffers because the system has to calculate more than you actually care about. Set it too low and accuracy suffers. Even when you're working with integers, going past DIGITS means it goes exponential.

3) Not so huge a downside, but also worth considering: REXX doesn't have complex number support at all. You have to simulate it with two variables. There's no way for the "Python number" system to intrinsically handle complexes.

So really, what you're looking at is unifying int/float/Decimal into a single data type, which is basically Decimal. I'd say merging int into that is a bad idea, but you can get most of what you want to achieve simply by encouraging the use of Decimal everywhere.

Maybe in some distant future version, the Python literal 1.234 will represent a Decimal rather than a float. (And then there'd be other consequences, like what you get when you divide one integer by another.) But until then, all you can really do is encourage people to explicitly use Decimal.

ChrisA

7:34 a.m.

On Wednesday, March 5, 2014 12:57:10 AM UTC-6, Chris Angelico wrote:

1) Performance. Huge huge performance hit. I can take Pike to OS/2, an unoptimized build on an unsupported platform, and absolutely cream a REXX program at any sort of computational work. By specifically working with integers rather than "numbers", I can run code blazingly fast.

2) Accuracy. With REXX, all arithmetic is governed by a single setting of NUMERIC DIGITS, which specifies how many digits of accuracy you want to preserve. (Python can improve granularity with separate contexts, but the same issue will still apply - contexts just let you use different NUMERIC DIGITS settings simultaneously in one program.)

hi Chris, good to hear from another Rexx programmer! Back in the day (during my 25 year tenure with IBM, I wrote thousands of scripts in Rexx for VM370 systems, and OS/2. I too still have OS/2 running. whoohoo!

Thanks for your comments. I would normally have agreed with you on this but not in this case. I do not think you are wrong, I just think you may have underestimated the power of decimal.Decimal /its phenomenal--really--!

Don't take the Rexx reference too much to heart, I just wanted folks to know where I got my influence on this thinking. Rexx was too slow... so was Decimal until 3.3, and then, boy has it taken off so-to-speak!

Implementation details are way off in the future of course, but I am thinking AI processing on this idea. The Python system will sense and optimize the decimal contexts (local and otherwise) so that things are balanced between speed and accuracy. And I would note, that speed is not as important for most python related aspects as user clarity and modern policy and account- ability. But, I'm not the least concerned for performance being a problem, because I have thoroughly tested decimal.Decimal and it flat screams, unlike our old friend Rexx.

Thanks again for your feedback on the idea. Please give it some more thought, and let me know what you think of AI for load/speed/accuracy balancing.

PS Have you considered the way things might have been if Palmisano had played ball with Bill Gates and Steve Ballmer back in the day... OS/2 would rule, gnu/linux might never have been invented period, and you and I might not be having this conversation today! ha! Go Big Blue.

marcus

9:36 a.m.

On Wed, Mar 5, 2014 at 6:34 PM, Mark H. Harris harrismh777@gmail.com wrote:

hi Chris, good to hear from another Rexx programmer! Back in the day (during my 25 year tenure with IBM, I wrote thousands of scripts in Rexx for VM370 systems, and OS/2. I too still have OS/2 running. whoohoo!

Don't take the Rexx reference too much to heart, I just wanted folks to know where I got my influence on this thinking. Rexx was too slow... so was Decimal until 3.3, and then, boy has it taken off so-to-speak!

Yes, that's true. But I've just done some performance testing. REXX on our OS/2 VM accounting server "Stanley" calculates 10000! in 14 seconds; Pike on the same hardware calculates 100000! in 4 seconds. (I didn't bother calculating 100000! in REXX, didn't feel like waiting.) In each case it was by this naive algorithm:

REXX: n=1; do i=1 to 10000; n=n*i; end
Pike: int n=1; for (int i=1;i<=100000;++i) n*=i;

Those were running on the same hardware, and going to a target an order of magnitude higher took a fraction of the time. That's what integer performance is like. (BTW, I set 'numeric digits' to something just a bit more than the target. REXX was pretty fast if I left digits low, and I could then look at the exponent to see what digits setting I needed. In Pike's case, the int type supports arbitrary precision anyway, so the net result is a fair comparison; I could write the digits out to a file and get the exact same result.)

From here on, I've switched computers to Sikorsky, who has more languages and more power than Stanley has. Timings here aren't technically comparable to the above ones, but they seem similar. I guess VirtualBox is doing a decent job of staying out of the way. Pike on Sikorsky:

gauge {int n=1; for (int i=1;i<=100000;++i) n*=i;}; (3) Result: 3.394805453

Okay. Now, Python.

Python 3.4.0rc1+ (default:a124b981a7a3, Mar 3 2014, 01:30:30) [GCC 4.7.2] on linux

def fac(top): t=time.time() n=1 for i in range(1,top+1): n*=i return time.time()-t fac(100000) 7.814447641372681

Roughly double Pike's time. (I suspect this may be largely because Pike has an optimization for machine-word integers. The difference is far starker if the algorithm used is less naive.) Now watch the result of a switch of data type.

def fac_d_maxprec(top): from decimal import Decimal, getcontext, MAX_PREC getcontext().prec=MAX_PREC t=time.time() n=Decimal("1") for i in range(1,top+1): n*=i return time.time()-t

(Note that the multiplication is still with ints. Doing all the multiplication with Decimals would mean a whole lot more calling of the constructor, which wouldn't be a fair comparison. If all of Python used Decimal, then range() would be returning Decimal.)

fac_d_maxprec(100000) 20.71300506591797

I also wrote an alternative version of the above which pre-calculates at low precision, then sets the precision to just enough, and recalculates. The timings were statistically identical to the above.

So there you are: A three to one difference. That's pretty amazing,
actually - the fact that it's not *far far* worse is a tribute to the
folks who gave us this highly optimized decimal.Decimal
implementation! But it's still dramatically slower than integer
calculations, and as you can see from the Pike version, the integer
figures can be cut a long way too.

I'm not sure that I want to give that up.

But, I'm not the least concerned for performance being a problem, because I have thoroughly tested decimal.Decimal and it flat screams, unlike our old friend Rexx.

Yes, that's true (although frankly, when I first met REXX on OS/2, it screamed compared to doing the job myself in C - sure, I could do integer calculations quicker in C, but if I wanted to go above the size of a long - 1<<32 - that was pretty hard), but the screaming isn't so impressive compared to a modern integer engine.

PS Have you considered the way things might have been if Palmisano had played ball with Bill Gates and Steve Ballmer back in the day... OS/2 would rule, gnu/linux might never have been invented period, and you and I might not be having this conversation today! ha! Go Big Blue.

Hmm, I don't think so. Linux had a rather different purpose, back then. But yeah, OS/2 was brilliant tech built in a hostile environment... I'd like to see some of its best parts lifted onto a Linux kernel, though (like the WorkPlace Shell - should be possible to run that on top of X11). Alas, most of OS/2 now is in the "was great in the 90s, but other things do the same job now" basket, and a closed-source system that boasts little above a GNU/Linux stack isn't really going to go anywhere much.

Still, it's hanging around. It's the best way I have for running 16-bit Windows programs, believe it or not. (And yes, I did try running Pastel Accounting under Wine. Win-OS/2 still beats Wine hands-down, probably because Wine developers don't see any reason to bother with supporting anything so ancient.)

ChrisA

8:10 a.m.

My previous reply got google-grouped, so let me try again.

From: Mark H. Harris harrismh777@gmail.com Sent: Tuesday, March 4, 2014 7:42 PM

The Python language might be changed to adopt the python number

concept for all math processing, unless explicitly modified. This goes somewhat beyond using decimal floating point as a default numerical type. It means using human numeric expressions that meet human expectation for numeric processing by default.

Different humans have different expectations in different situations. Trying to pick a single type that can handle all such expectations is an impossible dream.

Under the covers (whatever we mean by that) processing is handled by decimal.Decimal, unless explicitly modified. What does this mean for python users in general? Well, no more worrying about types at all... no ints, no floats, no rationals, no irrationals, no fractions, and certainly no binaries. In short, if its a human number, its a python number.

This is not really a meaningful concept. The types you dislike are not programming concepts that get in the way of human mathematics, they are human mathematical concepts. The integers, the rationals, and the reals all behave differently. And they're not the only types of numbers—Python handles complex numbers natively, and it's very easy to extend it with, say, quaternions or even arbitrary matrices that act like numbers.

The types that _are_ programming concepts—decimal and binary floats—are necessary, because you simply can't store real numbers in finite storage. And the very fact that they are inexact approximations means you can't ignore the types. For some uses, IEEE binary floats are best; for others, decimal floats are best; for others, fractions are best; for others, you even want to handle symbolic numbers like pi/2 exactly.

I am expecting that (just as in Rexx numbers) defining very clearly what

is a python number will be key for wide adaptation of the concept. But there should be no surprises for users, particularly average users. Anyone with a middle school expectation of a numeric format should be able to use python numbers without surprises.

Anyone with a middle school expectation will expect 1/3 to be a fraction—or, at least, something they can multiply by 3 to get exactly 1. Using an inexact decimal float instead of an inexact binary float is no improvement at all. Sure, it's an improvement in some _other_ cases, like 0.123, but if you want to deal with 1/3, today you can do so explicitly by using, say, Fraction(1, 3), while in your world it will no longer be possible.

And if you take the obvious way around that, you run into the same problem with 2 ** 0.5. Normal humans who write that would expect to be able to square it and get back 2, not an approximation to 2.

However, for advanced users the full interface should be available (as is the context for Decimal) through coding based on knowledge and experience, yet the default context for Decimal should be based on average users in most environments and use cases.

Is there a problem with the current default context?

I think just using Decimal as it is in Python today gets you everything you're asking for. Sure, you might want a nicer way to type them and repr them, which goes back to the previous thread, but why do you need to get rid of all of the other types as well?

11:29 a.m.

On 5 March 2014 08:10, Andrew Barnert abarnert@yahoo.com wrote:

From: Mark H. Harris harrismh777@gmail.com >

I am expecting that (just as in Rexx numbers) defining very clearly what is a python number will be key for wide adaptation of the concept. But there should be no surprises for users, particularly average users. Anyone with a middle school expectation of a numeric format should be able to use python numbers without surprises.

Anyone with a middle school expectation will expect 1/3 to be a fraction--or, at least, something they can multiply by 3 to get exactly 1. Using an inexact decimal float instead of an inexact binary float is no improvement at all.

I actually think that it is an improvement. Most people are surprised by the fact that just writing x = 0.1 causes a rounding error. Python only has dedicated syntax for specifying binary floats in terms of decimal digits meaning that there is no syntax for exactly specifying non integer numbers. I would say that that is clearly a sub-optimal situation.

I don't agree with Mark's proposal in this thread but I would like to have decimal literals e.g. 1.23d, and I would also use Fraction literals if available e.g. 1/3F.

Oscar

12:04 p.m.

On Wed, Mar 05, 2014 at 11:29:10AM +0000, Oscar Benjamin wrote:

I don't agree with Mark's proposal in this thread but I would like to have decimal literals e.g. 1.23d,

+1 on that. Although that will depend on how big a slowdown it causes to Python's startup time.

and I would also use Fraction literals if available e.g. 1/3F.

Out of curiosity, why F rather than f?

(By the way, I think it is somewhat amusing that Python not only has a
built-in complex type, but also *syntax* for creating complex numbers,
but no built-in support for exact rationals.)

-- Steven

12:30 p.m.

On 5 March 2014 12:04, Steven D'Aprano steve@pearwood.info wrote:

On Wed, Mar 05, 2014 at 11:29:10AM +0000, Oscar Benjamin wrote:

I don't agree with Mark's proposal in this thread but I would like to have decimal literals e.g. 1.23d,

+1 on that. Although that will depend on how big a slowdown it causes to Python's startup time.

If it is a significant slowdown then it can be a delayed import that only occurs when a decimal literal is first encountered. Applications that don't need decimal won't be slowed down. Applications that do would have had to import it anyway.

and I would also use Fraction literals if available e.g. 1/3F.

Out of curiosity, why F rather than f?

In C and some other languages the f suffix indicates a numeric literal that has type "float" e.g. "6.0f". You can use upper case there as well but the convention is lower case and to include the .0 when it's an integer. I just though that 3F looks sufficiently distinct from the way it's typically done in C.

Oscar

12:42 p.m.

On Wed, Mar 5, 2014 at 11:30 PM, Oscar Benjamin

oscar.j.benjamin@gmail.com wrote:

I just though that 3F looks sufficiently distinct from the way it's typically done in C.

3F would be about -16C wouldn't it?

(Not painting any bike sheds when it's that cold, thanks!)

I'm not sure I like the idea of tagging the end of the expression. Currently, the nearest Python has to that is the complex notation:

1+2j (1+2j)

And that works just fine when considered to be addition:

a=1 b=2j a+b (1+2j)

So really, what Python has is a notation for a j-suffixed float, meaning a complex number with no real part. You can't do that with Fraction:

a=2 b=3F a/b

What I'd prefer would be some modification of the division operator. We currently have two ways to divide an integer by an integer:

1234/10 123.4 1234//10 123

Adding a third operator, defined only between two integers, would be cleaner than tagging one of the integer literals. Exactly what that operator would be is hard to say, but I'm sure there's a combination that would look right and not be ambiguous.

1234-/-10 Fraction(617, 5)

I can't think of anything good, though.

ChrisA

12:56 p.m.

On 5 March 2014 12:42, Chris Angelico rosuav@gmail.com wrote:

On Wed, Mar 5, 2014 at 11:30 PM, Oscar Benjamin

oscar.j.benjamin@gmail.com wrote:

I just though that 3F looks sufficiently distinct from the way it's typically done in C.

3F would be about -16C wouldn't it?

(Not painting any bike sheds when it's that cold, thanks!)

I'm not sure I like the idea of tagging the end of the expression. Currently, the nearest Python has to that is the complex notation:

1+2j (1+2j)

And that works just fine when considered to be addition:

a=1 b=2j a+b (1+2j)

So really, what Python has is a notation for a j-suffixed float, meaning a complex number with no real part. You can't do that with Fraction:

a=2 b=3F a/b

From a syntactic perspective Python doesn't have syntax for general complex literals. There is only syntax for creating imaginary literals and it is trivial to create a complex number by adding a real and an imaginary one.

3F could create a Fraction with the integer value 3 so that a/b gives a rational number:

from fractions import Fraction as F a = 2 b = F(3) a/b Fraction(2, 3)

I don't understand why you say that can't be done.

Oscar

3:23 p.m.

On Wed, Mar 5, 2014 at 11:56 PM, Oscar Benjamin

oscar.j.benjamin@gmail.com wrote:

3F could create a Fraction with the integer value 3 so that a/b gives a rational number:

from fractions import Fraction as F a = 2 b = F(3) a/b Fraction(2, 3)

I don't understand why you say that can't be done.

(Also Steven who said the same thing.)

Uhh... brown-paper-bag moment. When I wrote up that post, I somehow blanked out the obvious fact that Fraction can happily represent an integer. Whoops...

As Julia Jellicoe said, my objection falls to the ground. Very well!

ChrisA

1:23 p.m.

On Wed, Mar 05, 2014 at 11:42:14PM +1100, Chris Angelico wrote:

I'm not sure I like the idea of tagging the end of the expression. Currently, the nearest Python has to that is the complex notation:

1+2j (1+2j)

And that works just fine when considered to be addition:

a=1 b=2j a+b (1+2j)

So really, what Python has is a notation for a j-suffixed float, meaning a complex number with no real part. You can't do that with Fraction:

a=2 b=3F a/b

Why not? If 3d is a Decimal with the value of 3, why couldn't 3F be a Fraction with the value of 3?

-- Steven

10:49 p.m.

Oscar Benjamin wrote:

In C and some other languages the f suffix indicates a numeric literal that has type "float" e.g. "6.0f". You can use upper case there as well but the convention is lower case and to include the .0 when it's an integer. I just though that 3F looks sufficiently distinct from the way it's typically done in C.

I'm not sure it's different enough; I've been looking at Java code recently that uses uppercase F.

An alternative would be to use 'r' or 'R' for 'rational', which would eliminate any chance of confusion.

-- Greg

7 Mar
7 Mar

8:39 p.m.

On Wed, Mar 5, 2014, at 7:30, Oscar Benjamin wrote:

In C and some other languages the f suffix indicates a numeric literal that has type "float" e.g. "6.0f". You can use upper case there as well but the convention is lower case and to include the .0 when it's an integer. I just though that 3F looks sufficiently distinct from the way it's typically done in C.

How about 3r?

On the subject of suffixes, it might be worth considering that they often use "d" for "double" as well. C# uses "m" (for money) for decimal.

5 Mar
5 Mar

1:39 p.m.

Steven D'Aprano:

I don't agree with Mark's proposal in this thread but I would like to have decimal literals e.g. 1.23d,

+1 on that. Although that will depend on how big a slowdown it causes to Python's startup time.

Startup time should not be a problem once http://bugs.python.org/issue19232 is dealt with.

Stefan Krah

3:38 p.m.

On 03/05/2014 06:04 AM, Steven D'Aprano wrote:

(By the way, I think it is somewhat amusing that
Python not only has a
built-in complex type, but also*syntax* for creating complex numbers,
but no built-in support for exact rationals.)

That is interesting.

I think Mark is correct in unifying numbers. And also adding in decimal features. The way it should actually be done is another thing. But having an up to date package that can be used is a very good start. (Thanks Mark!)

Mark describes an AI approach,, which I think he means having a internal representation that may change as needed depending on how a number can best be stored and calculated while still keeping it's accuracy.

Weather or not that approach is called Decimals is another thing. It might be called "Unified Numbers".. or just Numbers.

The point is for the internal representation to be an implementation detail the user doesn't need to worry about. And have Decimal features available by default.

If things are decided by use case, then I can't even think of a good enough argument against having decimal features available by default. The financial use cases are that overwhelming.

-Ron

10:36 p.m.

Steven D'Aprano wrote:

(By the way, I think it is somewhat amusing that
Python not only has a
built-in complex type, but also *syntax* for creating complex numbers,
but no built-in support for exact rationals.)

I gather that Guido's experiences with ABC have led him to believe such a feature would be an attractive nuisance. Rationals have some nasty performance traps if you use them without being fully aware of what you're doing.

-- Greg

11:24 p.m.

On 5 March 2014 22:36, Greg Ewing greg.ewing@canterbury.ac.nz wrote:

Steven D'Aprano wrote: >

*syntax* for creating complex numbers, but
no built-in support for exact rationals.)

I gather that Guido's experiences with ABC have led him to believe such a feature would be an attractive nuisance. Rationals have some nasty performance traps if you use them without being fully aware of what you're doing.

I've read statement's to that effect but they were specifically about
having rational as the *default* mode for integer division which I am
not suggesting.

It's already possible to opt-in to using Fractions so that rationals are used for division. Currently you have to import a module and write F(1, 7) or F('1/7') everywhere in your code. If it were possible to write 1/7F or 1/7r or whatever then I've personally written code where I would have used that so that it would be easier to read, would look more natural with syntax highlighting etc.

Oscar

6 Mar
6 Mar

11:59 p.m.

Coming to this very late, but I was interested in the first thread...

On 05Mar2014 23:04, Steven D'Aprano steve@pearwood.info wrote:

On Wed, Mar 05, 2014 at 11:29:10AM +0000, Oscar Benjamin wrote:

+1 on that. Although that will depend on how big a slowdown it causes to Python's startup time.

+1 on 123d decimal literals.

and I would also use Fraction literals if available e.g. 1/3F.

Out of curiosity, why F rather than f?

Well, for me, +1 on 123f for "IEEE float" literals. i.e. the floats in play in Python right now.

If ever we get ambiguity on how numbers are done, this may be useful in the transition or experiment phase.

So of course "F" is reasonable for fractions. But how hard does that make the grammar? Because the parser now has to grab the entire "1/3F" to construct the fraction. You can't just stick it in the lexer at that point.

Aside: Unless you do something obtuse, like make "3F" be a regular number with a special internal flag which is treated differently on the right hand side of a division operation: if encoutered, division now returns a Fraction instead of an int or float; this feels like something that could easily have nasty side effects.

And, speaking personally, a big -1 on Mark's "abstract all the numbers". Like other posters, I agree that a number's internal implementation/representation affects its semantics. You can't just wave a wand and say "it should all be abstract"; there will be side-effects.

You can wave a wand and say "from here on in this code, unadorned numbers make Decimals, not IEEE floats".

I'm +0.5 on something like:

from __future__ import decimal_floats

to imply making Decimals from unadorned literals. "__future__" may be the wrong pseudomodule name.

Not least, it makes testing the side-effects much easier because you can tweak a nice big chunk of code without editing it internally.

*syntax* for creating complex numbers,
but no built-in support for exact rationals.)

Harder to parse (see above), smaller use case maybe. It is not a bad idea, just not yet done...

Cameron Simpson cs@zip.com.au

The concept is interesting and well-formed, but in order to earn better than a 'C,' the idea must be feasible. --A Yale University management professor in response to Fred Smith's paper proposing reliable overnight delivery service. (Smith went on to found Federal Express Corp.)

7 Mar
7 Mar

12:17 a.m.

On Fri, Mar 7, 2014 at 10:59 AM, Cameron Simpson cs@zip.com.au wrote:

Aside: Unless you do something obtuse, like make "3F" be a regular number with a special internal flag which is treated differently on the right hand side of a division operation: if encoutered, division now returns a Fraction instead of an int or float; this feels like something that could easily have nasty side effects.

Thanks, now I can take the paper bag off my head. I'm not the only one!

I thought exactly what you say here, that 3F would have to be a magical type of integer. But actually, all it has to be is Fraction(3) and everything will work perfectly.

1/Fraction(3) Fraction(1, 3)

ChrisA

2:16 a.m.

On 7/03/2014 1:17 p.m., Chris Angelico wrote:

On Fri, Mar 7, 2014 at 10:59 AM, Cameron Simpson cs@zip.com.au wrote:

Aside: Unless you do something obtuse, like make "3F" be a regular number with a special internal flag

I thought exactly what you say here, that 3F would have to be a magical type of integer. But actually, all it has to be is Fraction(3) and everything will work perfectly.

I think Cameron was talking about using 'F' for both 'binary
floating point' *and* 'fraction', which would indeed lead to
madness.

-- Greg

2:58 a.m.

On 07Mar2014 15:16, Greg Ewing greg.ewing@canterbury.ac.nz wrote:

On 7/03/2014 1:17 p.m., Chris Angelico wrote:

On Fri, Mar 7, 2014 at 10:59 AM, Cameron Simpson cs@zip.com.au wrote:

Aside: Unless you do something obtuse, like make "3F" be a regular number with a special internal flag

I thought exactly what you say here, that 3F would have to be a magical type of integer. But actually, all it has to be is Fraction(3) and everything will work perfectly.

I think Cameron was talking about using 'F' for both 'binary
floating point' *and* 'fraction', which would indeed lead to
madness.

Well, only because I hadn't thought through to having it be a literal

Cameron Simpson cs@zip.com.au

The reasonable man adapts himself to the world; the unreasonable one persists in trying to adapt the world to himself. Therefore all progress depends on the unreasonable man. - George Bernard Shaw

12:32 a.m.

On 2014-03-06 23:59, Cameron Simpson wrote:

Coming to this very late, but I was interested in the first thread...

On 05Mar2014 23:04, Steven D'Aprano steve@pearwood.info wrote:

On Wed, Mar 05, 2014 at 11:29:10AM +0000, Oscar Benjamin wrote:

+1 on that. Although that will depend on how big a slowdown it causes to Python's startup time.

+1 on 123d decimal literals.

and I would also use Fraction literals if available e.g. 1/3F.

Out of curiosity, why F rather than f?

Well, for me, +1 on 123f for "IEEE float" literals. i.e. the floats in play in Python right now.

If ever we get ambiguity on how numbers are done, this may be useful in the transition or experiment phase.

So of course "F" is reasonable for fractions. But how hard does that make the grammar? Because the parser now has to grab the entire "1/3F" to construct the fraction. You can't just stick it in the lexer at that point.

I think it would be confusing if there were "f" for "float" and "F" for "fraction". How about "r" for "rationals"?

Aside: Unless you do something obtuse, like make "3F" be a regular number with a special internal flag which is treated differently on the right hand side of a division operation: if encoutered, division now returns a Fraction instead of an int or float; this feels like something that could easily have nasty side effects.

And, speaking personally, a big -1 on Mark's "abstract all the numbers". Like other posters, I agree that a number's internal implementation/representation affects its semantics. You can't just wave a wand and say "it should all be abstract"; there will be side-effects.

You can wave a wand and say "from here on in this code, unadorned numbers make Decimals, not IEEE floats".

I'm +0.5 on something like:

from __future__ import decimal_floats

to imply making Decimals from unadorned literals. "__future__" may be the wrong pseudomodule name.

Not least, it makes testing the side-effects much easier because you can tweak a nice big chunk of code without editing it internally.

*syntax* for creating complex numbers,
but no built-in support for exact rationals.)

Harder to parse (see above), smaller use case maybe. It is not a bad idea, just not yet done...

12:45 a.m.

On Fri, Mar 7, 2014 at 11:32 AM, MRAB python@mrabarnett.plus.com wrote:

I think it would be confusing if there were "f" for "float" and "F" for "fraction". How about "r" for "rationals"?

Since the string prefixes are all case insensitive, I'd be very much surprised if f and F did different things.

<pedantic>Python doesn't have a syntax for
creating complex numbers; it
has a syntax for creating imaginary numbers.</pedantic>

<pedantic>
>>> type(1j)
<class 'complex'>
</pedantic>ChrisA

2:52 a.m.

On 07Mar2014 00:32, MRAB python@mrabarnett.plus.com wrote:

On 2014-03-06 23:59, Cameron Simpson wrote:

Coming to this very late, but I was interested in the first thread...

On 05Mar2014 23:04, Steven D'Aprano steve@pearwood.info wrote:

On Wed, Mar 05, 2014 at 11:29:10AM +0000, Oscar Benjamin wrote:

and I would also use Fraction literals if available e.g. 1/3F.

Out of curiosity, why F rather than f?

Well, for me, +1 on 123f for "IEEE float" literals. i.e. the floats in play in Python right now. [...] So of course "F" is reasonable for fractions. But how hard does that make the grammar? Because the parser now has to grab the entire "1/3F" to construct the fraction. You can't just stick it in the lexer at that point.

I think it would be confusing if there were "f" for "float" and "F" for "fraction". How about "r" for "rationals"?

Cameron Simpson cs@zip.com.au

If you do not read the paper, you are uninformed. If you do read the paper, you are misinformed. - Mark Twain

5 Mar
5 Mar

8:32 p.m.

From: Oscar Benjamin oscar.j.benjamin@gmail.com

Sent: Wednesday, March 5, 2014 3:29 AM

On 5 March 2014 08:10, Andrew Barnert abarnert@yahoo.com wrote:

From: Mark H. Harris harrismh777@gmail.com

I am expecting that (just as in Rexx numbers) defining very clearly what is a python number will be key for wide adaptation of the concept. But there should be no surprises for users, particularly average users. Anyone with a middle school expectation of a numeric format should be able to use python numbers without surprises.

Anyone with a middle school expectation will expect 1/3 to be a fraction--or, at least, something they can multiply by 3 to get exactly 1. Using an inexact decimal float instead of an inexact binary float is no improvement at all.

I actually think that it is an improvement. Most people are surprised by the fact that just writing x = 0.1 causes a rounding error. Python only has dedicated syntax for specifying binary floats in terms of decimal digits meaning that there is no syntax for exactly specifying non integer numbers. I would say that that is clearly a sub-optimal situation.

I don't agree with Mark's proposal in this thread but I would like to have decimal literals e.g. 1.23d, and I would also use Fraction literals if available e.g. 1/3F.

I agree with you completely on that. That's kind of my point—your suggestion is almost the _opposite_ of the proposal in this thread. You want to make it easier for people to use the appropriate type for each use case; he wants to eliminate that choice entirely so you never have to make it. The latter would be nice if it were doable, but it's not even possible in principle. So the former is the best choice.

9:13 p.m.

On 03/05/2014 02:32 PM, Andrew Barnert wrote:

I don't agree with Mark's proposal in this thread but I would like to have decimal literals e.g. 1.23d, and I would also use Fraction literals if available e.g. 1/3F.

I agree with you completely on that. That's kind of my point—your suggestion is almost the_opposite_ of the proposal in this thread. You want to make it easier for people to use the appropriate type for each use case; he wants to eliminate that choice entirely so you never have to make it. The latter would be nice if it were doable, but it's not even possible in principle. So the former is the best choice.

It's also a matter of how far in the future you are looking. If he waited until later to propose something like this, it most likely wouldn't get in. I'm not sure he's expecting a firm answer now, but probably is hoping for a "hey lets look into this more and see if it's really possible" kind of maybe. For that, it's good to start early.

On the near term, adding literal syntax's as you describe here would get all the pieces into python. Then an import from future could enable what Mark is thinking. He really needs to flesh out the details first before we can make any objective opinions about how it would actually work.

The way I see it, with a unified number type, we will still need context like api's to get the more specialised behaviours. The difference may be a decorator on a function that specifies some subset of number types to use rather than notating each literal and casting each value.

```
@numbers(int, decimal)
def acounting_foo(...):
...
# using ints for integers
# using decimal for reals
...
```

That just a guess... maybe Mark can give some examples how what he's proposing would work. ?

Cheers, Ron

6 Mar
6 Mar

12:40 a.m.

On Wednesday, March 5, 2014 3:13:29 PM UTC-6, Ron Adam wrote: >

It's also a matter of how far in the future you are looking. If he waited

until later to propose something like this, it most likely wouldn't get in.

I'm not sure he's expecting a firm answer now, but probably is hoping for

a "hey lets look into this more and see if it's really possible" kind of maybe. For that, it's good to start early.

hi Ron, yes, correct. Something like this would require huge effort, determined strategy, and possibly years of planning. Its not going to happen over-night, but in stages could be brought to fruition given some thought and a lot of dialogue.

kind regards,

5 Mar
5 Mar

1:21 p.m.

On Wed, Mar 05, 2014 at 12:10:18AM -0800, Andrew Barnert wrote:

Anyone with a middle school expectation will expect 1/3 to be a fraction—or, at least, something they can multiply by 3 to get exactly 1.

Not a very good example -- that happens to work for Python floats (which are C doubles under the hood):

py> (1/3)*3 == 1 True py> (1/3 + 1/3 + 1/3) == 1 True

But it *does not work* with Decimal, at least not with the default
precision:

py> from decimal import Decimal py> (Decimal(1)/3)*3 Decimal('0.9999999999999999999999999999')

Decimal is not a panacea! It does not eliminate floating point issues. Between 1 and 100, there are 32 Decimal numbers that fail the test that (1/n)*n == 1, and only two floats.

Between 1 and 100, there are only four floats where 1/(1/n) does not equal n: 49 93 98 and 99. In comparison, there are 46 such failing Decimals, including 6 7 and 9.

-- Steven

6 Mar
6 Mar

12:30 a.m.

On Wednesday, March 5, 2014 2:10:18 AM UTC-6, Andrew Barnert wrote:

The types you dislike are not programming concepts that get in the way of

human mathematics, they are human mathematical concepts. The integers, the rationals, and the reals all behave differently. And they're not the only types of numbers—Python handles complex numbers natively, and it's very easy to extend it with, say, quaternions or even arbitrary matrices that act like numbers.

hi Andrew, your response reminds me of that scene from 'Star Trek: First
Contact,'

when Data was getting a little lippy with her eminence The Borg (the one
who is many)

(he was trying to explain to her why she doesn't exist, since her ship was
destroyed
just after coming through the chronometric particle vortex--time travel--)
and she
responded, "You think so three dimensionally... "

When asked about the Borg hierarchy she stated simply, "You imply disparity where none exists..."

Now, that was one hot cybernetic babe. My point is philosophical, really. I do not dislike numerical types. I simply do not believe they are necessary. In fact, Rexx (in a simple way) and other languages too for that matter, have demonstrated that types are superficial paradigms in symbolic thought that can quite readily be eliminated for most AI applications given enough memory, and enough time. Fortunately, memory is becoming more plentiful (less expensive) and time is shrinking away through processor speed, pipe-lining, and software engineering like decimal.Decimal.

Keep in mind that I am NOT proposing the elimination of numbers, I am
proposing the
abstraction of *python numbers (yes, all of them...) *in such a way that
programmers and
specialists, and accountants and grade school teachers may use Python
without having
to be computer scientists (all of the work happens under the covers,
written by someone
like me, or you) and everything at the top is simple and human readable;
for them.

Yes, its work. And, its doable. A *number *should be no more convoluted
in a modern age
of symbol set processing than any other abstract idea (just because of
arbitrary limits
and abstraction paradigms). Unifying *python numbers *requires a high
order paradigm
shift for sure; but the concept is absolutely doable.

kind regards,

5 Mar
5 Mar

1:56 p.m.

On Tue, Mar 04, 2014 at 07:42:28PM -0800, Mark H. Harris wrote:

The idea of *python number* means that there
are no types, no limits, no
constraints, and that all *python numbers *are dynamic. The upper level
syntax is also very simple; all* python numbers *are simply human.

What makes this a "python number"? In what way are they "dynamic"?

My influence for this preference is rooted in the
Rexx programming
language; attributed to Mike Cowlishaw, former IBM fellow. The Rexx
programming language is dynamic, and has no types. To put it more
succinctly for those of you who have not used Rexx, the only data type
is a string of characters (that's it). *Rexx numbers* are simply those
strings of characters that may be interpreted as a valid *Rexx number.*

I haven't used Rexx, but I have used Hypertalk, which worked the same way. If you don't care about performance, it can work quite well.

The Python language might be changed to adopt the
*python number*
concept for *all math processing*, unless explicitly modified.

Well, there's a bit of a problem here. Numbers in Python are not just
used for maths processing. They're also used for indexing into lists, as
keys in dicts, for bitwise operations, for compatibility with external
libraries that have to interface with other languages, as flags, etc.
For some of these purposes, we *really do* want to distinguish between
ints and floats that happen to have the same value:

mylist[3] # allowed mylist[3.0] # not allowed

Now, you might argue that this distinction is unnecessary, but it runs quite deep in Python. You'd need to change that philosophy for this idea to work.

This goes somewhat beyond using decimal floating point as a default numerical type. It means using human numeric expressions that meet human expectation for numeric processing by default.

I don't understand what that means, unless it means that you want Python to somehow, magically, make all the unintuitive issues with floating point to disappear. Good luck with that one.

If you want that, Decimal is not the answer. It would have to be a Rational type, like Fraction, although even that doesn't support surds. Fractions have their own problems too. Compare the status quo:

py> 3**0.5 1.7320508075688772

with a hypothetical version that treats all numbers as exact fractions:

py> 3**0.5 Fraction(3900231685776981, 2251799813685248)

Which do you think the average person using Python as a calculator would prefer to see?

And another issue: before he invented Python, Guido spent a lot of time working with ABC, which used Fractions as the native number type. The experience soured him on the idea for nearly two decades. Although Guido has softened his stance enough to allow the fractions module into the standard library, I doubt he would allow Fractions to become the underlying implementation of numbers in Python.

The problem is that fractions can be unbounded in memory, and some simple operations become extremely inefficient. For example, without converting to float, which is bigger?

Fraction(296, 701) Fraction(355, 594)

For many purposes, the fact that floats (whether binary or decimal) have finite precision and hence introduce rounding error is actually a good thing. Compare:

py> 1e-300 + 1e300 1e+300

versus fractions:

py> from fractions import Fraction as F
py> F(10)**-300 + F(10)**300
Fraction(100000000000000000000000000000000000000000000000000000000000000
000000000000000000000000000000000000000000000000000000000000000000000000
000000000000000000000000000000000000000000000000000000000000000000000000
000000000000000000000000000000000000000000000000000000000000000000000000
000000000000000000000000000000000000000000000000000000000000000000000000
000000000000000000000000000000000000000000000000000000000000000000000000
000000000000000000000000000000000000000000000000000000000000000000000000
000000000000000000000000000000000000000000000000000000000000000000000000
0000000000000000000000000000000001, 100000000000000000000000000000000000
000000000000000000000000000000000000000000000000000000000000000000000000
000000000000000000000000000000000000000000000000000000000000000000000000
000000000000000000000000000000000000000000000000000000000000000000000000
0000000000000000000000000000000000000000000000000)

There are *very few* applications where such precision is needed or
wanted. The performance hit in calculating such excessively precise
numbers when the user doesn't need it will be painful.

The same applies to Decimal, although to a lesser extent since Decimals do have a finite precision. Unlike fractions, they cannot grow without limit:

py> Decimal("1e-300") + Decimal("1e300") Decimal('1.000000000000000000000000000E+300')

-- Steven

6 Mar
6 Mar

1:06 a.m.

On Wednesday, March 5, 2014 7:56:46 AM UTC-6, Steven D'Aprano wrote: >

On Tue, Mar 04, 2014 at 07:42:28PM -0800, Mark H. Harris wrote:

*python number* means that
there are no types, no limits,
no
constraints, and that all *python numbers *are dynamic. The upper level
syntax is also very simple; all* python numbers *are simply human.

What makes this a "python number"? In what way are they "dynamic"?

At this point in the discussion please try not to think "implementation,"
rather think
conceptually in an open framework where anything is possible. Dynamic in
terms
of *python numbers *is not that far off from the concept of dynamic name
binding used
everywhere else in python.

In terms of unifying numbers as *python numbers *which needs to be
defined at some
point, the idea of dynamic binding means first that numeric symbol sets
have no type,
and are not 'bound' until run-time, if at all. Consider:

What is this? ===> PI / 2

Is it two numbers with a divisor,? or a Decimal and an int with a
divisor,? or a *python number ?*

Or, is it a string of characters that are readily recognizable to humans that means 'the top of the unit circle," or half_of_PI whatever I mean by that, or '1.57079632679489661923132169163975144209858469968755291048747229615390820314'

Dynamic means the symbols sets are bound (name-wise, abstraction-wise) only at run-time, and with some 'smarts' as it were (AI) so that the right things happen under the covers (not magic) because the machine has been taught to think, as it were, about numbers and symbols the way normal humans do (in the kitchen, at high school, at the university, in science and business).

Another way to think of this Steven is to think of 'pretty print' on the
TI 89 Platinum edition
graphical programming calculator (my personal favorite). Anyone who plays
with it the first time
is frustrated because *numbers *are symbolized instead of printed out in
strings of numerals. So
if I enter {1} [/] {3} and press enter, I get 1/3 <====

Yes, I can press evaluate and get .33333333333333333 but 1/3 is just as good a number as any other number (and what type is it???) PI / 2 is another example. That's a great number and I don't really need to see a numeric list of digits to use it, do I... and what is its type??? who cares.

Another very simple (and not quite good enough) example is the pdeclib file I just created over the past couple of days. I have a need within the module to use PI or half_PI for some method or function (but I don't need the user to know that, and I don't want to calculate it over and over, and I want it within accuracy for the context on demand)... so what? I created __gpi__ global PI. It is reset to Decimal(0) if the dscale(n) precision context changes. Then, and function that requires PI of half_PI can check first to see if it is cached as __gpi__ and if so pull it back, or if not calculate it. If the context does not change than I half some whopping value of PI for the duration of the context (but, and this is the big but, if I don't need it --- then it never gets bound).

Binding for *python numbers * must be dynamic, typeless, limitless, and no
constraints. If I enter a
formula of symbols into python (whatever I mean by that) python just
handles it dynamically just
like the TI 89--- if I want explicit control of the processing, I am free
to ask for that too, just like
the TI 89.

By the by, complex numbers are no different what-so-ever, nor fractions
either, nor any other *number.*
The system needs to be able to parse symbol sets intelligently (Alonzo
Church, Alan Turning, and John
McCarthy dreamed of this, but had no way to accomplish it in their day) and
then dynamically bind
the naming / representation to the abstraction (or implementation) that
makes sense for that symbol
set in 'that' context, for 'that' problem. This is not simply business as
usual... it requires AI, it requires
a severe paradigm shift of a very high order.

Kind regards, marcus

2:21 a.m.

> >

Then, and function that requires PI of half_PI can check first to see if it is cached as __gpi__ and if so pull it back, or if not calculate it. If the context does not change than I half some whopping value of PI for the duration of the context (but, and this is the big but, if I don't need it --- then it never gets bound).

um, sorry, I wixed my mords... please forgive,

"Then, and only then, a function that requires PI or half_PI

will first check to see if __gpi__ has been cached and if so pull it
back, or if not calculate it (once). If the context
does not change then the function can take half of some whopping value of
PI, for the duration of the context, and
use that, but if not needed for the context will never bind it !

marcus

3:47 a.m.

I wonder if we need yet another list, python-speculative-ideas? Or python-waxing-philosophically? I've got a feeling were straying quite far from the topic of the design of even Python 4, let alone what could possibly happen in Python 3.

-- --Guido van Rossum (python.org/~guido)

4:05 a.m.

On Wednesday, March 5, 2014 9:47:24 PM UTC-6, Guido van Rossum wrote: >

I wonder if we need yet another list, python-speculative-ideas? Or python-waxing-philosophically?

hi Guido, no probably not. I just got to thinking again about *number
*again
as
I was thinking about decimal floating point as default. Its the
mathematician in me; go
for the general case, and that would be a type-less unified *number *system.

For the near case design issues the following realist ideas might be considered in order:

1) decimal literal like 1.23d

2) default decimal floating point with a binary type 1.23b

3) type-less unified *number *yes, probably way out there into the future

kind regards, marcus

5:56 a.m.

Do you actually have a degree in math, or do you just remember your high school algebra? The numbers in math usually are quite strictly typed: whole theories only apply to integers or even positive integers, other things to all reals but not to complex numbers. (And what about quaternions? Or various infinities? :-)

On Wed, Mar 5, 2014 at 8:05 PM, Mark H. Harris harrismh777@gmail.comwrote:

> >

On Wednesday, March 5, 2014 9:47:24 PM UTC-6, Guido van Rossum wrote: >

I wonder if we need yet another list, python-speculative-ideas? Or python-waxing-philosophically?

hi Guido, no probably not. I just got to thinking again about *number
*again
as
I was thinking about decimal floating point as default. Its the
mathematician in me; go
for the general case, and that would be a type-less unified *number *
system.

For the near case design issues the following realist ideas might be considered in order:

1) decimal literal like 1.23d

2) default decimal floating point with a binary type 1.23b

3) type-less unified *number *yes, probably way out there into the future

kind regards, marcus

-- --Guido van Rossum (python.org/~guido)

8:14 a.m.

On Wednesday, March 5, 2014 11:56:02 PM UTC-6, Guido van Rossum wrote: >

Do you actually have a degree in math, or do you just remember your high school algebra?

hi Guido, ouch. Its a story, glad you asked. My first trip to college
1974-1977

(UMKC) was to study EE; minor in mathematics. I completed my math course
work
(Calc 1-5, Diff Eq, Linear Algebra, Theory of Stats and Prob, &c) ... all
(A). Not
that it matters, because in 1977 IBM made me an offer in their CE
department at
Kansas City... so I joined IBM for the next 25 years and did not complete
my EE
degree... but, I did complete my mathematics training. Of course, prior to
UMKC
I attended a high school that offered calculus, math analysis, trig... and
intro to
linear algebra. So by the time I joined IBM I got the math twice. So,
yeah, I know
what I'm doing. I am an amateur mathematician today, computer scientist,
and
computer hobbyist. While at IBM I was a staff software engineer, Tampa,
Chicago,
Atlanta, and at the lab at Rochester MN (where I left IBM in 2002; yet
I still dwell there). Education is a life long commitment and I continue to
study math, comp sci, music
and philosophy. I just completed my course work for the MDiv degree at
Bethel Seminary.

Back in the day, I added the scientific and transcendental math functions
to the Rexx
library for the internal VM370 systems, because Rexx (like python) also had
no decimal
floating point math package. So, yeah, its one of the things I know how to
do, and its one
of those things that most people never think about; but I've noticed that
they appreciate
having it once the work is done. My pdeclib package on PyPI is in infancy
stage, probably
has bugs (yet nobody has complained yet) and pdeclib will have to mature
there for some time.
But that has really nothing to do with whether we continue to use IEEE 754
1985 floats & doubles
nor whether we discuss default decimal floating point arithmetic nor
whether we discuss a unified
*python number *sytem (sometime in the distant future) that would make |
allow common average
ordinary people to leverage mathematics in computer science without having
to understand the
underlying mechanics of implementation including but not limited to types.

We might agree to stick with the discussion, if you're are willing, and
stay away from *ad hominem*
attacks, nor credential bashing. A person either knows what they are doing,
or they don't. And if
they are willing to contribute, why knock it, or them?

The numbers in math usually are quite strictly typed: whole theories only apply to integers or even positive integers, other things to all reals but not to complex numbers.

Everyone keeps making the points noted above! Yes, I know... concur~
Guido, all data types in most computer high level languages
are all very strictly statically bound also. So what? You know this better
than anyone, because you have taken so much abuse from trolls about it over
the
years. We all know that the best way to handle name binding is dynamically.
And that
was a paradigm shift, was it not? Go take a look at Rexx. It has NO types.
None.
Not at the surface, anyway. Everything to the user of the language is a
string. This is
even true for most of "object" Rexx too. *Numbers *are just strings of
characters that
are parsed and interpreted by Rexx as valid *Rexx Numbers*. It works. There
is no
real point in arguing that its a bad idea, because it served the Rexx
community for
many years... even now, Rexx is not dead. Yes, under the covers (not magic)
a complex
number is going to be handled differently than a real number. Yeah, what
is your point?
They are handled differently in my TI 89 too... so what, I change the
context on the 89
and now I'm doing complex number math (not often, I might add). If I set
the context for
reals, then now I'm doing real math... it really is not that difficult.
Think about this for just
a minute. I have used complex math exactly three times in my life. I used
it in high school
to study the concept in math analysis. I used it in my EE classes...
electrical engineers
get a lot out of complex numbers. And I used it when I was interested in
the Mendlebrot
set many years ago when it was cool to plot the famous fractal on early cga
screens.
When was the last time you used complex numbers? When was the last time a
bank, or
a hospital, or Montgomery Wards used complex numbers? If someone needs
complex
numbers, Python could change the context "dynamically" and move through the
problem set.

Why should the user need to "manually" change the context if (AI) could
change it for them
on-the-fly? Just try to get a feel for the question, and stop with trying
to beat me up on
my credentials.

(And what about quaternions? Or various infinities? :-)

What about quaternions? If you extend the complex number system you change the context. That's not hard... You would not expect the context for "reals" processing to be the same for the context of processing three dimensional space with pairs of complex numbers, would you? Python does complex number processing now. But that is a very specialized category of use case that requires special context and (under the covers) typing relevant to complex number pairs. The context can change "dynamically" to suit the problem set, that's all I'm saying.

I am not saying in all of my dreaming here that typing is not important (down under). Python is called an object based language, yes? Not object oriented, right? But we all know that down under the covers (not magic) python uses Classes and instances of classes--objects. We don't pretend that what makes object based languages work is magic? do we? Unifying the number system on a computer does not have to be grounded in paradigms that serviced the industry (and academy) for the past 40 years; mostly because memory was expensive and processing was slow. I suggest that if memory had been cheap, back in the day, and processing had been very fast (as it is today) IEEE 754 1985 floats and doubles would never have been used. We would have used Decimal floating points right from the get-go. Initially, that really is all I'm asking for on the outset--- lets move to default decimal floating point arithmetic for real numbers processing.

In the future I am thinking about systems of languages that interact with
human speech; understanding
*spoken number* just like they will understand *symbol number.* There is
really no reason (other
than paradigm) that this would not be possible either.

Otherwise, Guido, we might all of us just continue to use C and code up our statically bound types by hand using nothing but int, long, float, and double.

Its time to innovate.

Kind regards, BDfL,

marcus

5:48 p.m.

On Thu, Mar 6, 2014 at 12:14 AM, Mark H. Harris harrismh777@gmail.comwrote:

On Wednesday, March 5, 2014 11:56:02 PM UTC-6, Guido van Rossum wrote: >

Do you actually have a degree in math, or do you just remember your high school algebra?

hi Guido, ouch. Its a story, glad you asked. My first trip to college 1974-1977 [...]

I must have hit a nerve -- I wasn't requesting a transcript! :-) A simple "I dropped out of math in college to pursue a career in programming" would have been sufficient (my own story is pretty similar :-).

We might agree to stick with the discussion, if
you're are willing, and
stay away from *ad hominem*
attacks, nor credential bashing. A person either knows what they are
doing, or they don't. And if
they are willing to contribute, why knock it, or them?

I was asking because I am having a hard time getting your point, and it feels like you keep repeating it without clarifying it.

The numbers in math usually are quite strictly typed: whole theories only

apply to integers or even positive integers, other things to all reals but not to complex numbers.

Everyone keeps making the points noted above! Yes, I know... concur~ Guido, all data types in most computer high level languages are all very strictly statically bound also.

I'm not sure that that description applies to Python; it sure doesn't match
how I *think* about numbers in Python.

So what? You know this better than anyone, because you have taken so much abuse from trolls about it over the years.

Actually, I haven't -- I can smell a troll a mile away and just mute the conversation.

We all know that the best way to handle name binding is dynamically.

That's your opinion. There are pros and cons and both static and dynamic binding have their place.

And that was a paradigm shift, was it not?

I've never claimed such a hyperbole.

Go take a look at Rexx. It has NO types. None.
Not at the surface, anyway. Everything to the user of the language is a
string. This is
even true for most of "object" Rexx too. *Numbers *are just strings of
characters that
are parsed and interpreted by Rexx as valid *Rexx Numbers*.

Why do you italicize this?

It works. There is no real point in arguing that its a bad idea, because it served the Rexx community for many years... even now, Rexx is not dead. Yes, under the covers (not magic) a complex number is going to be handled differently than a real number. Yeah, what is your point? They are handled differently in my TI 89 too... so what, I change the context on the 89 and now I'm doing complex number math (not often, I might add).

By "context", here, do you mean some specific state in the computer, or are you referring to a different way of thinking about it (i.e. in your head)?

If I set the context for reals, then now I'm doing real math... it really is not that difficult.

So I suppose depending on how you set the context, the square root of -1 either returns a complex number, or raises an error? And comparisons are right out once your context is set for complex numbers? Or are they still allowed when the imaginary part is exactly zero? (What if it is nearly zero?)

Think about this for just a minute. I have used complex math exactly three times in my life. I used it in high school to study the concept in math analysis. I used it in my EE classes... electrical engineers get a lot out of complex numbers. And I used it when I was interested in the Mendlebrot set many years ago when it was cool to plot the famous fractal on early cga screens. When was the last time you used complex numbers? When was the last time a bank, or a hospital, or Montgomery Wards used complex numbers? If someone needs complex numbers, Python could change the context "dynamically" and move through the problem set. Why should the user need to "manually" change the context if (AI) could change it for them on-the-fly? Just try to get a feel for the question, and stop with trying to beat me up on my credentials.

I still don't understand the question. Can you be more precise instead of using more rhetoric? Is your point just that you don't want to have to deal with complex numbers? That's fine, and it's already (mostly) how Python works -- you must use a complex literal or the cmath module before any complex numbers appear (the exception is that x**y returns a complex number when x is negative and y is not a whole number -- but arguably that's the best answer).

(And what about quaternions? Or various infinities? :-) >

What about quaternions? If you extend the complex number system you change the context. That's not hard...

So, by context you really seem to be referring to some specific state of the program. Perhaps you are even talking about extending the existing notion of numeric context used by Python's Decimal type. But it's not very clear because "you change the context" is also a common term for changing the topic of a discussion.

You would not expect the context for "reals" processing to be the same for

the context of processing three dimensional space with pairs of complex numbers, would you?

Actually I'm not sure I *need* a context for this. In Python things like
this are usually just solved through a combination of operator overloading
and dynamic binding, so that e.g. x*y can be a number when x and y are
numbers, but it can mean a vector product of some sort if x and/or y are
vectors.

Python does complex number processing now. But that is a very specialized category of use case that requires special context and (under the covers) typing relevant to complex number pairs. The context can change "dynamically" to suit the problem set, that's all I'm saying.

I'm sorry, but I am *still* unsure about what you mean by "context".
Python
does not need a context object to switch between complex and real number
processing -- it's all done through operator overloading.

I am not saying in all of my dreaming here that typing is not important (down under). Python is called an object based language, yes? Not object oriented, right?

Actually, modern Python is considered object-oriented. I called it
object-based in a *very* early stage, when there was no class statement in
the language. But IIRC it was added before the first public release (or
soon after), although another round of improvements in this area happened
in the early 2000s (eradicating most of the differences between classes
defined by C and Python code).

But we all know that down under the covers (not magic) python uses Classes and instances of classes--objects. We don't pretend that what makes object based languages work is magic? do we?

"We"?

Unifying the number system on a computer does not have to be grounded in paradigms that serviced the industry (and academy) for the past 40 years; mostly because memory was expensive and processing was slow. I suggest that if memory had been cheap, back in the day, and processing had been very fast (as it is today) IEEE 754 1985 floats and doubles would never have been used. We would have used Decimal floating points right from the get-go. Initially, that really is all I'm asking for on the outset--- lets move to default decimal floating point arithmetic for real numbers processing.

Now you've got a specific proposal and we can discuss pros and cons. This is a topic that comes up occasionally and it is indeed a pretty interesting trade-off. I suspect the large (and fast-growing) SciPy community would prefer to keep binary floating point, because that's what their libraries written in Fortran or C++ want, and extra binary/decimal conversions at the boundaries would just slow things down. But I'm really speculating here -- maybe they don't need high-performance conversion between binary and decimal, since (perhaps) they may do their bulk I/O in C++ anyway. Or maybe it varies per application.

In the future I am thinking about systems of
languages that interact with
human speech; understanding
*spoken number* just like they will understand *symbol number.* There is
really no reason (other
than paradigm) that this would not be possible either.

This is the kind of talk that I wish we could keep out of the python-ideas list. (Except for the symbolic algebra, for which various third-party Python libraries already exist.)

Otherwise, Guido, we might all of us just continue to use C and code up our statically bound types by hand using nothing but int, long, float, and double.

Its time to innovate.

Rhetoric again.

Kind regards, BDfL,

marcus

-- --Guido van Rossum (python.org/~guido)

10:23 p.m.

On Thursday, March 6, 2014 11:48:53 AM UTC-6, Guido van Rossum wrote: {snip}

hi Guido, thanks for your responses, your candor, and your kindness; I think dialogue will work, and I promise to keep the rhetoric to a bare minimum, if at all. :)

There really are only two questions I am going to try to answer here, and some of the answer goes back to Stefan's (and others) questions as well, so I hope its helpful.

So, by context you really seem to be referring to some specific state of the program. Perhaps you are even talking about extending the existing notion of numeric context used by Python's Decimal type. But it's not very clear because "you change the context" is also a common term for changing the topic of a discussion.

```
This is what I mean by "context," for this entire discussion, mostly
```

having to do with Decimal and the "context manager" and these two lines:

```
getcontext()
with localcontext(ctx=None) as context_manager
Allow me to back up a bit... the pdeclib module was not difficult
```

for me because of the mathematics (no brag) I can code transcendental functions in my sleep. The hard part of that project last week was coming to grips with the context manger, and the parameterization of context attributes in the Decimal class. Back in the day when I did this work for Rexx at IBM I used NUMERIC DIGITS and NUMERIC FUZZ to emulate (in a crude way) what Decimal is doing correctly with its context, with the context manager. I credit Oscar Benjamin for helping me get up to speed with this concept (and I am so glad he did too, because its key for this entire discussion.

```
Before we go on, allow me to explain my italics of *Number *and
```

*PythonNumber.* These are
each an abstract base class. They are a class that is never instantiated,
but from which all
other "numbers" are derived as objects. (more below)

I'm sorry, but I am *still* unsure about what
you mean by "context".
Python does not need a context object to switch between complex and real
number processing -- it's all done through operator overloading.

```
(more on this below, but I am thinking beyond overloading/
```

actually, virtual functions and templates)

Actually, modern Python is considered object-oriented.

```
Thank you. This has everything to do with this discussion, but
```

allow me to take a humorous break and relate an anecdote... I have had numerous discussions with T.R. and C.A and S.D. and others where in the end python was object based. Ha! Nope, you have seen it right here, the BDfL says its "object oriented," and that's the end of the argument! (ok, tiny rhetoric)

>

Unifying the

number system on a computer does not have to be grounded in paradigms that serviced the industry (and academy) for the past 40 years;

```
{snip}
```

>

Now you've got a specific proposal and we can discuss pros and cons. This is a topic that comes up occasionally and it is indeed a pretty interesting trade-off. I suspect the large (and fast-growing) SciPy community would prefer to keep binary floating point, because that's what their libraries written in Fortran or C++ want, and extra binary/decimal conversions at the boundaries would just slow things down. But I'm really speculating here -- maybe they don't need high-performance conversion between binary and decimal, since (perhaps) they may do their bulk I/O in C++ anyway. Or maybe it varies per application.

```
Here is the main response for this discussion, and I will endeavor
```

to keep the rhetoric down, I promise. This may even be easier to accomplish in 3.x without breaking anyone, or any third party software either; depends

```
In my view numbers are objects (like circles, squares and triangles;
```

in polymorphism discussions).
Numbers are nothing more than objects which have "contexts" just like
circles and squares and triangles
are nothing more than objects that have "contexts". In the following
discussion think in terms of object
oriented (as in Grady Booch) rather than any specific language--- and
certainly not python syntax. um,
think C++ for now, if you must think of any. A *Number *is an abstract
base class with "context" attributes
and pure virtual functions--- it will never be instantiated---and *PythonNumber
*will derive from that; also
an abstract base class. Here is the Class hierarchy I am proposing:

*Number*

```
*PythonNumber*
Decimal
Real <======= this is the default
BinaryIEEE85
Real_b
&c (...) omitted for clarity
Complex
Quaternion
Rational
Fraction
&c--------------------------------------------------
Well, there it is. The idea is that any function or method that
```

takes a *Number* reference
or a *PythonNumber *reference does not have to interrogate what type of
*Number* it is... it just
works (why?) glad you asked, because of polymorphism and a three pointer
hop down a virtual
function table!
If a function or method gets a *Number *parm in as in method(num)
then to get the pretty
print representation of the number is simply num.pretty_print(), or
to get an evaluation
of the number is num.eval(), or to get the string rep of the number is
num.str()... and on and on.
The point is that other parts of the system no longer need to know what
"type" of number object
it is, the right code gets called because of virtual functions, operator
overloading, and the beautiful
virtual function table (all under the covers, not MAGIC, but polymorphism).
Python at present is disjointed when it comes to number as a
concept. In my view numbers
need to be objects, and the Class hierarchy (see my simplified version
above) needs to unify numbers
across the python boards (so to speak) so that "types" are only important
to the implementors, and
so that adding new "types" is easy, concise, and coherent.

```
I realize that this suggestion is over the top in terms of size and
```

weight (Stefan's concern above).
But not really, when you think of conceptualizing the unification of
numbers under *Number *and
*PythonNumber *. There may be other kinds of numbers too, besides these...
like maybe *GmpyNumber*
of *BC_Number*, or others.

```
The bottom line is that we have an object oriented language in front
```

of us that has the software engineering advantages which would allow this kind of hierarchy and conceptualization of a unified number system. Let's leverage our technology for mutual advantage across python communities?

Thanks for your consideration,

Sincerely,

Kind regards,

marcus

10:41 p.m.

On Fri, Mar 7, 2014 at 9:23 AM, Mark H. Harris harrismh777@gmail.com wrote:

```
Well, there it is. The idea is that any
function or method that
```

takes a Number reference or a PythonNumber reference does not have to interrogate what type of Number it is... it just works (why?) glad you asked, because of polymorphism and a three pointer hop down a virtual function table!

That's way too concrete for Pythonic style :)

All you need to do is have each numeric type implement the same operations, and there you are, as the Lord Chancellor said, out of your difficulty at once!

And that's what we already have. You can add two numbers and they'll simply add. In fact, a C++ style "hop down a virtual function table" couldn't handle that. I could make a class hierarchy like you describe, and have each one have an add() method or operator+(), but somewhere along the line, something still has to cope with the fact that it could be given any sort of number - there has to be code someplace that handles the adding of a (Decimal) Real and a Real_b, because some day, someone's going to do it. (The correct response might be to raise TypeError, on the basis that that action merges two separate inaccuracies. But more likely, the correct response is to convert one of them to the other type.)

What does your proposed hierarchy offer that numbers.Number doesn't?

ChrisA

11:58 p.m.

On Thursday, March 6, 2014 4:41:08 PM UTC-6, Chris Angelico wrote:

>

What does your proposed hierarchy offer that numbers.Number doesn't?

hi Chris, numbers.Number is the right concept, but not the correct relationship... in sixty's vernacular, its the right hoo hoo but the wrong tah tah...

Here is the numbers.Number hierarchy:

CLASSES builtins.object Number Complex Real Rational Integral

Compare numbers.Number with my proposal. (I am not wanting to go into semantics at this point, but the numbers.Number model is not unified and has problems. If it were ok, we would not be having this discussion. ) The age old question, "How's that working for you?" Well, not too well.

Chris, its not just the virtual function table, its also operator overloading, and its policy handled by (AI) that intelligently decides what to do if someone tries to add a real_b with a real_d. But, it should not do this:

from pdeclib import * x=1 y=Decimal(.1) x+y Decimal('1.10000000000000000555111512312578270211816')

The policy hidden under the abstraction way down in the guts of the Class hierarchy should make the decision to do this:

====== RESTART ===== from pdeclib import * x=1 y=.1

def add_(x, y): return d(x)+d(y)

add_(x, y) Decimal('1.1')

Please don't pick at this. The point is not the code or the syntax. The point is that the policy in a correctly related Class hierarchy can handle somebody asking the system to add a 1.23b with a 1.23d , and if setup correctly, will just work.

The idea is to unify numbers, so that efficient intelligent processing can occur without "types" or "limits".

I'll say more later... after we see how folks respond.

marcus

7 Mar
7 Mar

12:29 a.m.

On Fri, Mar 7, 2014 at 10:58 AM, Mark H. Harris harrismh777@gmail.com wrote:

But, it should not do this:

from pdeclib import * x=1 y=Decimal(.1) x+y Decimal('1.10000000000000000555111512312578270211816')

The policy hidden under the abstraction way down in the guts of the Class hierarchy should make the decision to do this:

====== RESTART ===== from pdeclib import * x=1 y=.1

def add_(x, y): return d(x)+d(y)

add_(x, y) Decimal('1.1')

Please don't pick at this. The point is not the code or the syntax. The point is that the policy in a correctly related Class hierarchy can handle somebody asking the system to add a 1.23b with a 1.23d , and if setup correctly, will just work.

If you had a literal syntax "0.1d" meaning Decimal("0.1"), this would be solved. The only reason your code seems to work is that you're rounding off your binary floats instead of using them with as much accuracy as they have. You're deceiving yourself that str(float('0.1')) seems to round-trip, and therefore that's the right way to get a decimal value from a float. But with floats completely out of the picture, there's no need to do any of this.

x=1 y=Decimal(".1") # y=.1d x+y Decimal('1.1')

So... I'm +1 for adding a literal syntax for decimals. +1 for adding a stating-the-default for floats (do it straight away, then code that uses it can be backported all the way even when there's a change of default). +0.5 for adding a syntax for fractions; support in principle but the devil's in the details. -1 for trying to unify everything.

ChrisA

1:52 a.m.

From: Chris Angelico rosuav@gmail.com

Sent: Thursday, March 6, 2014 4:29 PM

So... I'm +1 for adding a literal syntax for decimals. +1 for adding a stating-the-default for floats (do it straight away, then code that uses it can be backported all the way even when there's a change of default).

This makes sense if you also add an optional suffix for binary floats at the same time. Otherwise, it would be confusing to talk about the "default" when there's no way to make it explicit, and it would delay any potential change of the default by at least one version, if not more.

Of course there's a lot of room for bikeshedding the suffix. The "f" suffix from C implies 32-bit float as opposed to 64-bit double, which is obviously wrong. The "d" suffix from C might be confusing as distinguishing "binary, not decimal". Maybe "b"?

+0.5 for adding a syntax for fractions; support in principle but the devil's in the details.

What details, other than bikeshedding the exact suffix? If 3r == fraction.Fraction(3), we're done, right?

2:12 a.m.

On 2014-03-07 01:52, Andrew Barnert wrote:

From: Chris Angelico rosuav@gmail.com

Sent: Thursday, March 6, 2014 4:29 PM

So... I'm +1 for adding a literal syntax for decimals. +1 for adding a stating-the-default for floats (do it straight away, then code that uses it can be backported all the way even when there's a change of default).

This makes sense if you also add an optional suffix for binary floats at the same time. Otherwise, it would be confusing to talk about the "default" when there's no way to make it explicit, and it would delay any potential change of the default by at least one version, if not more.

Of course there's a lot of room for bikeshedding the suffix. The "f" suffix from C implies 32-bit float as opposed to 64-bit double, which is obviously wrong. The "d" suffix from C might be confusing as distinguishing "binary, not decimal". Maybe "b"?

Calling the floating-point class "float" implies 32-bit float as opposed to 64-bit double, doesn't it? :-)

+0.5 for adding a syntax for fractions; support in principle but the devil's in the details.

What details, other than bikeshedding the exact suffix? If 3r == fraction.Fraction(3), we're done, right?

2:26 a.m.

From: MRAB python@mrabarnett.plus.com

Sent: Thursday, March 6, 2014 6:12 PM

On 2014-03-07 01:52, Andrew Barnert wrote:

Of course there's a lot of room for bikeshedding the suffix. The "f" suffix from C implies 32-bit float as opposed to 64-bit double, which is obviously wrong. The "d" suffix from C might be confusing as distinguishing "binary, not decimal". Maybe "b"?

Calling the floating-point class "float" implies 32-bit float as opposed to 64-bit double, doesn't it? :-)

Sure. It's not as strong an implication as an "f" suffix, because many languages (and applications) have types named float, while "0.3f" is a lot more C-family-specific. Which means "float" wasn't a perfect choice—but it was still probably the best choice. And it's certainly possible that "f" is the best choice for the suffix despite this problem. If there were an obvious best answer that had no problems, there wouldn't be any room for bikeshedding.

4:16 a.m.

On Fri, Mar 7, 2014 at 12:52 PM, Andrew Barnert abarnert@yahoo.com wrote:

From: Chris Angelico rosuav@gmail.com

Sent: Thursday, March 6, 2014 4:29 PM

So... I'm +1 for adding a literal syntax for decimals. +1 for adding a stating-the-default for floats (do it straight away, then code that uses it can be backported all the way even when there's a change of default).

This makes sense if you also add an optional suffix for binary floats at the same time. Otherwise, it would be confusing to talk about the "default" when there's no way to make it explicit, and it would delay any potential change of the default by at least one version, if not more.

Yep. That's what I mean (when I said "float"s up there, I meant the current type 'float', aka binary floating point). The suffix won't have any effect; it'll be like u"string" in Py3, explicitly stating the default, and added for the exact same reason.

+0.5 for adding a syntax for fractions; support in principle but the devil's in the details.

What details, other than bikeshedding the exact suffix? If 3r == fraction.Fraction(3), we're done, right?

Precisely that detail. Is it 3r? 3F? Something else? Would it look tidier as a prefix instead of a suffix? But if that can be resolved, it'd be good to have a syntax for fractions.

I'm only +0.5 on that, as rationals aren't as big an advantage over the status quo as decimal is. It's a lot less common to see:

Fraction(1/3) Fraction(6004799503160661, 18014398509481984)

than the oddities of decimal.Decimal construction that we're seeing here. Having a numeric suffix for decimal literals will be much more beneficial to the language, imo; if the bikeshedding of Fraction literals is problematic, I'd not be against doing decimal (and binary float) tags one version, and leaving Fraction for another version. ('Course, it might all work out perfectly, in which case great! Add 'em all at once.)

ChrisA

12:17 a.m.

On Thursday, March 6, 2014 4:41:08 PM UTC-6, Chris Angelico wrote:

What does your proposed hierarchy offer that numbers.Number doesn't? >

I just had one more thought along this line; consider this:

from pdeclib import *

s1=sqrt(2.01) s1 Decimal('1.41774468787578244511883132198668766452744') s2=sqrt(d(2.01)) s2 Decimal('1.41774468787578252029556185427085779261123')

s1**2
Decimal('2.00999999999999978683717927196994423866272')
s2**2
Decimal('2.01000000000000000000000000000000000000000')

Which one is right, s1 or s2 ?

Well, clearly s1 is wrong. And yet, s1 was coded by giving the system a normal human number, at a human level, and very innocently s1 is really badly broken but not immediately noticed until we try to square it...

s2 on the other hand is correct, but we had to go through some hoops to make sure that the system received the correct type. The system should be able to make this determination without the user having to determine types.

marcus

12:38 a.m.

On Fri, Mar 7, 2014 at 11:17 AM, Mark H. Harris harrismh777@gmail.com wrote:

Well, clearly s1 is wrong. And yet, s1 was coded by giving the system a normal human number, at a human level, and very innocently s1 is really badly broken but not immediately noticed until we try to square it...

s2 on the other hand is correct, but we had to go through some hoops to make sure that the system received the correct type. The system should be able to make this determination without the user having to determine types.

Once again, the problem occurs only because you're using a float literal, and 2.01 can't perfectly round-trip.

Decimal(2.01) Decimal('2.0099999999999997868371792719699442386627197265625')

That's pretty much the value you had (I have a few more digits there, the square rooting and squaring probably cost some accuracy), so I'd say Decimal correctly round-tripped. The problem is the conversion from string (source code) to float to Decimal. You think that str()ing a float gives you a better round-trip, but that works only because you're using relatively small numbers.

Hmm. Is the rounding done by float.__str__() an attractive nuisance? Would it be better to show exactly what's stored, if only to remove the temptation to treat str(f) as "more correct" than f?

ChrisA

2:08 a.m.

On Thursday, March 6, 2014 6:38:34 PM UTC-6, Chris Angelico wrote:

Once again, the problem occurs only because you're using a float

literal, and 2.01 can't perfectly round-trip.

Decimal(2.01) Decimal('2.0099999999999997868371792719699442386627197265625')

That's pretty much the value you had (I have a few more digits there, the square rooting and squaring probably cost some accuracy),

hi Chris, yes, you are completely missing the point. We are talking past each other. I DO NOT need you to explain to me why the is working just like its supposed to... its broken and should not work this way !

If you write a number on paper do you write down '2.01' ?

Do you write down d(2.01) ?

Do you write down 2.01d ?

(or) do you write down 2.01 ?

When you punch a number into your TI89 do you punch in {'} {2} {.} {0} {1} {'} ?

(or) do you punch in {2} {.} {0} {1} ?

When I want a square-root from python I want to enter s1=sqrt(2.01)

<====== this is normal
(why?) glad you asked, because that is the human normal thing to do.

Python numbers don't work correctly, because the underlying design is not correct. Everyone sees it, nobody really likes it, but everyone wants to avoid the problem like the proverbial ostrich (only worse , because this ostrich wants me to believe that everything is fine, " Don't pay any attention to that man behind the current... I... am the great and powerful OZ"

marcus

3:29 a.m.

On 06Mar2014 18:08, Mark H. Harris harrismh777@gmail.com wrote:

On Thursday, March 6, 2014 6:38:34 PM UTC-6, Chris Angelico wrote: Once again, the problem occurs only because you're using a float

literal, and 2.01 can't perfectly round-trip.

Decimal(2.01) Decimal('2.0099999999999997868371792719699442386627197265625')

That's pretty much the value you had (I have a few more digits there, the square rooting and squaring probably cost some accuracy),

hi Chris, yes, you are completely missing the point. We are talking past each other. I DO NOT need you to explain to me why the is working just like its supposed to... its broken and should not work this way !

If you write a number on paper do you write down '2.01' ?

Do you write down d(2.01) ?

Do you write down 2.01d ?

(or) do you write down 2.01 ?
[...]
When I want a square-root from python I want to enter s1=sqrt(2.01)

<====== this is normal

So essentially you are saying: when I write Python code I do not want to have to put in all this special notation to get the actual value which I naturally mean.

And if you write 2.01, you mean 2 + 1/100, without roundoff because you do not want to be working in a system which would round that off. i.e. a Decimal internal representation because you're writing a number in base 10.

To my mind you want a Python _parser_ mode, because inside python, given the various numeric classes abounding, the actual _mechanisms_ already exist.

So really you want to have a way of saying to the parser: in the code, 2.01 is not transformed into an IEEE float with some loss of precision, it is transformed into a Decimal floating point number, with no loss of precision.

There are, it seems to me, two basic ways to approach this.

The easy one is give Python something like:

from __the_far_far_future__ import Decimal_numbers

and have decimal literals converted into those instead of Python "floats".

The alternative is the implement a Python class, let us call it "AbtractNumber", which stores the literal text you put in the programme code "2.01" and converts it, _when used_, to a suitable python type with the right semantics.

Either would get you code where:

```
x = 2.01
```

is stored without loss of precision, and may get you the behaviour you're after.

Which of these more closely matches your desires?

Please try to be precise. Examples don't quite do it without some rather precise accompanying lagunage which says what specific things the example is meant to illustrate.

If one of my two suggestions above is very close to what you're after, please present an example where one suggestion does what you want and where the other would not, with an explaination of why.

Cameron Simpson cs@zip.com.au

Careful and correct use of language is a powerful aid to straight thinking, for putting into words precisely what we mean necessitates getting our own minds quite clear on what we mean. - W.I.B. Beveridge

10:57 a.m.

On Thu, Mar 06, 2014 at 06:08:28PM -0800, Mark H. Harris wrote:

If you write a number on paper do you write down '2.01' ? Do you write down d(2.01) ? Do you write down 2.01d ?

Sometimes I write down 2.01 subscript 10. That's just a different way of spelling 2.01d.

(That's the trouble with rhetorical questions. Sometimes people will give an answer other than what you were expecting.)

(or) do you write down 2.01 ?

When you punch a number into your TI89 do you punch in {'} {2} {.} {0} {1} {'} ? (or) do you punch in {2} {.} {0} {1} ?

For what it's worth, the TI-89 stores numbers using a decimal floating point format.

-- Steven

10:39 a.m.

On Fri, Mar 07, 2014 at 11:38:34AM +1100, Chris Angelico wrote:

Hmm. Is the rounding done by float.__str__() an attractive nuisance?

It's not *rounding* precisely. Starting in Python 3.1, the float
__repr__ will display the shortest decimal string that doesn't change
the float's value.

Reading the issue tracker history for this change is informative:

http://bugs.python.org/issue1580

To the guys who worked on that, I take my hat off to you all.

-- Steven

11:05 a.m.

On Fri, Mar 7, 2014 at 9:39 PM, Steven D'Aprano steve@pearwood.info wrote:

On Fri, Mar 07, 2014 at 11:38:34AM +1100, Chris Angelico wrote:

Hmm. Is the rounding done by float.__str__() an attractive nuisance?

It's not *rounding* precisely. Starting in Python 3.1, the float
__repr__ will display the shortest decimal string that doesn't change
the float's value.

Reading the issue tracker history for this change is informative:

http://bugs.python.org/issue1580

To the guys who worked on that, I take my hat off to you all.

That effect, yes. Let's call it "the magic of float.__str__", because it really is pretty amazing.

But it's still post-processing magic. It means that strings appear to round-trip through floats, as long as you're a long way within the available precision; but as soon as you do operations, that ceases to be the case. I think it's great for display, but is putting that into __repr__ (at least, they do appear to be the same) an attractive nuisance, in that it encourages people to treat float("...") as a true representation?

Don't get me wrong, it's a really *awesome* feature. It just happens
to have been partially responsible for this thread, which I think is
approaching 300 posts.

ChrisA

11:06 a.m.

On Fri, Mar 7, 2014 at 10:05 PM, Chris Angelico rosuav@gmail.com wrote:

Don't get me wrong, it's a really *awesome*
feature. It just happens
to have been partially responsible for this thread, which I think is
approaching 300 posts.

Whoops, no. This one's only approaching 100. It's the "should midnight be true or false" that's approaching 300. Sorry!

ChrisA

2:59 p.m.

Chris Angelico writes:

That effect, yes. Let's call it "the magic of float.__str__", because it really is pretty amazing.

But it's still post-processing magic. It means that strings appear to round-trip through floats, as long as you're a long way within the available precision; but as soon as you do operations, that ceases to be the case. I think it's great for display, but is putting that into __repr__ (at least, they do appear to be the same) an attractive nuisance, in that it encourages people to treat float("...") as a true representation?

What makes you think they need more encouragment?

Seriously, as one data point, I don't think having more "human" representations encourages me the think of floating point results as the product of arithmetic on real numbers. I don't think anybody who knows how tricky "floating point" arithmetic can be is going to be fooled by the "pretty eyes" of a number represented as "2.0" rather than "1.99999999999999743591".

3:09 p.m.

On Sat, Mar 8, 2014 at 1:59 AM, Stephen J. Turnbull stephen@xemacs.org wrote:

Seriously, as one data point, I don't think having more "human" representations encourages me the think of floating point results as the product of arithmetic on real numbers. I don't think anybody who knows how tricky "floating point" arithmetic can be is going to be fooled by the "pretty eyes" of a number represented as "2.0" rather than "1.99999999999999743591".

Fair enough. I just remember reading, back in my really REALLY early days with GW-BASIC, an explanation of why 3.2# (the hash made it double-precision) came out as whatever-it-did. Went into a full explanation of the nature of binary floating point, and the issue was forced to your attention because just about _any_ value that wasn't a neat multiple of a (negative) power of two would do that.

You can lead a programmer to docs, but you can't make him understand.

ChrisA

3:45 a.m.

On Thu, Mar 06, 2014 at 04:17:55PM -0800, Mark H. Harris wrote:

I just had one more thought along this line; consider this:

from pdeclib import * s1=sqrt(2.01) s1 Decimal('1.41774468787578244511883132198668766452744') s2=sqrt(d(2.01)) s2 Decimal('1.41774468787578252029556185427085779261123')

s1**2
Decimal('2.00999999999999978683717927196994423866272')
s2**2
Decimal('2.01000000000000000000000000000000000000000')

If you skip the conversion to Decimal, you actually get the right answer using floats:

py> (2.01**0.5)**2
2.01

So the problem here isn't the binary float, but that Decimal
by default has *too much precision* and consequently it ends
up keeping digits that the user doesn't care about:

py> from decimal import Decimal as D
py> (D.from_float(2.01)**D("0.5"))**2
Decimal('2.009999999999999786837179272')

Floats have about 14 significant base-10 figures of precision (more in base-2), so if we tell Decimal to use the same, we should get the same result:

py> import decimal
py> ct = decimal.getcontext()
py> ct.prec = 14
py> (D.from_float(2.01)**D("0.5"))**2
Decimal('2.0100000000000')

Decimal is not a panacea. Both Decimal and binary floats have the same limitations, they just occur in different places for different numbers. All floating point numbers have these same issues. Fixed point numbers have different issues, rationals have their own issues, and symbolic computations have a different set of issues.

Computer maths is a leaky abstraction. No matter what you do, how you implement it, the abstraction leaks. Not even Mathematica can entirely hide the fact that it is computing rather than performing a Platonic ideal of mathematics.

-- Steven

4:05 a.m.

On Thursday, March 6, 2014 9:45:47 PM UTC-6, Steven D'Aprano wrote:

Decimal is not a panacea. Both Decimal and binary floats have the same limitations, they just occur in different places for different numbers. All floating point numbers have these same issues. Fixed point numbers have different issues, rationals have their own issues, and symbolic computations have a different set of issues.

```
hi Steven, yes, I concur. My primary objection (for immediate
```

concern) is that we have now a very fast module for doing decimal floating point math with less of the issues you speak of. Doing decimal floating point math by default, or at a minimum, providing a way to enter decimal numbers 1.234d would do a lot to alleviate much of the perceived difficulty with float issues.

>

Computer maths is a leaky abstraction. No matter what you do, how you implement it, the abstraction leaks. Not even Mathematica can entirely hide the fact that it is computing rather than performing a Platonic ideal of mathematics.

```
Again, I concur. Leaky is one thing... but being mired in 40-year-old
```

paradigms because IEEE 754 1985 floats |doubles are so entrenched just doesn't make sense to me, and should be corrected.

```
I am not insisting on Guido's "sweeping" reform. I'm just trying to
```

make like a little easier for number munching and get folks to be forward thinking about moving to a decimal based floating point system of number for python by default---sometime.

```
Thank you for your response, Steven.
```

1:32 a.m.

From: Mark H. Harris harrismh777@gmail.com Sent: Thursday, March 6, 2014 2:23 PM

Here is the Class hierarchy I am proposing:

Number PythonNumber Decimal Real <======= this is the default BinaryIEEE85 Real_b &c (...) omitted for clarity Complex Quaternion Rational Fraction &c--------------------------------------------------

This kind of hierarchy doesn't work. Most importantly, you can have complex decimal floats and complex binary floats. This is why complex is a class template rather than a class in C++. Also, this is ignoring the numerical tower—rationals are a specialization of reals, just as floats are.

Well, there it is. The idea is that any function or method that takes a Number reference or a PythonNumber reference does not have to interrogate what type of Number it is... it just works (why?) glad you asked, because of polymorphism and a three pointer hop down a virtual function table!

99% of the time you don't need these abstract base classes at all, because duck typing and operator overloading. But when you need it, it's already there, and does everything you're asking for. See below.

If a function or method gets a Number parm in as in method(num) then to get the pretty print representation of the number is simply num.pretty_print(), or to get an evaluation of the number is num.eval(), or to get the string rep of the number is num.str()...

We already have the first and third as str() and repr(), and I'm not sure what the second is supposed to do. What does evaluating a number return other than the number itself?

Python at present is disjointed when it comes to number as a concept. In my view numbers need to be objects, and the Class hierarchy (see my simplified version above) needs to unify numbers across the python boards (so to speak) so that "types" are only important to the implementors, and so that adding new "types" is easy, concise, and coherent.

This is all just plainly false. Numbers are objects. There is a hierarchy of abstract base classes for them—done correctly—in the numbers module. And they handle everything you're asking for, except for one thing: There are no ABCs to distinguish binary vs. decimal floats. But I doubt that's useful for anything. What kind of code can you imagine that does need to distinguish, say, float vs. decimal.Decimal, but doesn't need to distinguish decimal.Decimal from mylib.MyDecimal? They're both inexact floating-point representations of reals.

If the motivation for your whole idea is that you want a class hierarchy like the one in the numbers module but you didn't know that it already existed, then we're done.

2:09 a.m.

Mark, it feels like you do not understand Python well enough to be able to make sweeping proposals about its reform. Hopefully the responses you are getting will help you make more informed proposals in the future.

On Thu, Mar 6, 2014 at 2:23 PM, Mark H. Harris harrismh777@gmail.comwrote:

> >

On Thursday, March 6, 2014 11:48:53 AM UTC-6, Guido van Rossum wrote: {snip}

hi Guido, thanks for your responses, your candor, and your kindness; I think dialogue will work, and I promise to keep the rhetoric to a bare minimum, if at all. :)

There really are only two questions I am going to try to answer here, and some of the answer goes back to Stefan's (and others) questions as well, so I hope its helpful.

So, by context you really seem to be referring to some specific state of the program. Perhaps you are even talking about extending the existing notion of numeric context used by Python's Decimal type. But it's not very clear because "you change the context" is also a common term for changing the topic of a discussion.

```
This is what I mean by "context," for this entire discussion,
```

mostly having to do with Decimal and the "context manager" and these two lines:

```
getcontext()
with localcontext(ctx=None) as context_manager
Allow me to back up a bit... the pdeclib module was not difficult
```

for me because of the mathematics (no brag) I can code transcendental functions in my sleep. The hard part of that project last week was coming to grips with the context manger, and the parameterization of context attributes in the Decimal class. Back in the day when I did this work for Rexx at IBM I used NUMERIC DIGITS and NUMERIC FUZZ to emulate (in a crude way) what Decimal is doing correctly with its context, with the context manager. I credit Oscar Benjamin for helping me get up to speed with this concept (and I am so glad he did too, because its key for this entire discussion.

```
Before we go on, allow me to explain my italics of *Number *and
```

*PythonNumber.* These are
each an abstract base class. They are a class that is never instantiated,
but from which all
other "numbers" are derived as objects. (more below)

I'm sorry, but I am *still* unsure about
what you mean by "context".
Python does not need a context object to switch between complex and real
number processing -- it's all done through operator overloading.

```
(more on this below, but I am thinking beyond overloading/
```

actually, virtual functions and templates)

Actually, modern Python is considered object-oriented.

```
Thank you. This has everything to do with this discussion, but
```

allow me to take a humorous break and relate an anecdote... I have had numerous discussions with T.R. and C.A and S.D. and others where in the end python was object based. Ha! Nope, you have seen it right here, the BDfL says its "object oriented," and that's the end of the argument! (ok, tiny rhetoric)

>

Unifying the

number system on a computer does not have to be grounded in paradigms that serviced the industry (and academy) for the past 40 years;

```
{snip}
```

>

Now you've got a specific proposal and we can discuss pros and cons. This is a topic that comes up occasionally and it is indeed a pretty interesting trade-off. I suspect the large (and fast-growing) SciPy community would prefer to keep binary floating point, because that's what their libraries written in Fortran or C++ want, and extra binary/decimal conversions at the boundaries would just slow things down. But I'm really speculating here -- maybe they don't need high-performance conversion between binary and decimal, since (perhaps) they may do their bulk I/O in C++ anyway. Or maybe it varies per application.

```
Here is the main response for this discussion, and I will endeavor
```

to keep the rhetoric down, I promise. This may even be easier to accomplish in 3.x without breaking anyone, or any third party software either; depends

```
In my view numbers are objects (like circles, squares and
```

triangles; in polymorphism discussions).
Numbers are nothing more than objects which have "contexts" just like
circles and squares and triangles
are nothing more than objects that have "contexts". In the following
discussion think in terms of object
oriented (as in Grady Booch) rather than any specific language--- and
certainly not python syntax. um,
think C++ for now, if you must think of any. A *Number *is an abstract
base class with "context" attributes
and pure virtual functions--- it will never be instantiated---and *PythonNumber
*will derive from that; also
an abstract base class. Here is the Class hierarchy I am proposing:

*Number*

```
*PythonNumber*
Decimal
Real <======= this is the default
BinaryIEEE85
Real_b
&c (...) omitted for clarity
Complex
Quaternion
Rational
Fraction
&c--------------------------------------------------
Well, there it is. The idea is that any function or method that
```

takes a *Number* reference
or a *PythonNumber *reference does not have to interrogate what type of
*Number* it is... it just
works (why?) glad you asked, because of polymorphism and a three pointer
hop down a virtual
function table!
If a function or method gets a *Number *parm in as in
method(num) then to get the pretty
print representation of the number is simply num.pretty_print(), or
to get an evaluation
of the number is num.eval(), or to get the string rep of the number is
num.str()... and on and on.
The point is that other parts of the system no longer need to know what
"type" of number object
it is, the right code gets called because of virtual functions, operator
overloading, and the beautiful
virtual function table (all under the covers, not MAGIC, but polymorphism).
Python at present is disjointed when it comes to number as a
concept. In my view numbers
need to be objects, and the Class hierarchy (see my simplified version
above) needs to unify numbers
across the python boards (so to speak) so that "types" are only important
to the implementors, and
so that adding new "types" is easy, concise, and coherent.

```
I realize that this suggestion is over the top in terms of size and
```

weight (Stefan's concern above).
But not really, when you think of conceptualizing the unification of
numbers under *Number *and
*PythonNumber *. There may be other kinds of numbers too, besides
these... like maybe *GmpyNumber*
of *BC_Number*, or others.

```
The bottom line is that we have an object oriented language in
```

front of us that has the software engineering advantages which would allow this kind of hierarchy and conceptualization of a unified number system. Let's leverage our technology for mutual advantage across python communities?

Thanks for your consideration,

Sincerely,

Kind regards,

marcus

-- --Guido van Rossum (python.org/~guido)

2:19 a.m.

On Thursday, March 6, 2014 8:09:02 PM UTC-6, Guido van Rossum wrote:

Mark, it feels like you do not understand Python well enough to be able to

make sweeping proposals about its reform.

hi Guido, ouch.

You can't be serious. You have not commented yet... are you happy with this:

from decimal import Decimal a=Decimal(1) b=Decimal(.1) a+b Decimal('1.100000000000000005551115123') <==== does this not bother you at all ?

```
... even though we both know why its doing this, doesn't it
bother you
```

a little?

marcus

2:53 a.m.

From: Mark H. Harris harrismh777@gmail.com Sent: Thursday, March 6, 2014 6:19 PM

[snipping out of order]

>>> from decimal import Decimal

a=Decimal(1) b=Decimal(.1) a+b Decimal('1.100000000000000005551115123') <==== does this not bother you at all ?

... even though we both know why its doing this, doesn't it bother you a little?

That's not a problem with Python's number system, it's that Decimal(.1) is not the right way to write what you want.

The only solution without changing Python is to train end-users to write something correct, like Decimal('.1').

The obvious solution for changing Python is to make it easier to create Decimal numbers correctly and/or harder to create them incorrectly. For example, a decimal suffix, as already proposed before this thread, would completely solve the problem:

>>> a = 1d >>> b = .1d >>> a+b 1.1d

Of course the exact suffix (or other syntax) is up for bikeshedding, as is the possibility of one day changing the default from binary floats to decimal floats, but other than those trivial details, tada. But there's no need for anything more radical, like some amorphous idea to "unify Python numbers".

On Thursday, March 6, 2014 8:09:02 PM UTC-6, Guido van Rossum wrote:

Mark, it feels like you do not understand Python well enough to be able to make sweeping proposals about its reform.

hi Guido, ouch.

You proposed that Python should handle numbers in an OO way, with numbers being real objects, instances of classes, with a hierarchy including abstract base classes; all of this is already there in Python.

You went off on a long digression about how you could implement this using the details of C++-style inheritance, when Python has a completely (and more powerful) different solution to inheritance that has already been used to solve this problem.

You proposed some complicated AI-based solution to solve the problem of using separate number classes in a single expression, even though Python (almost exactly like C++, in this case) has already solved that problem with operator overloading.

(And note that Python is flexible enough that third-party libraries can easily insert new types like quaternions, matrices, symbolic expressions, etc. into the hierarchy in a way that's transparent to end users. I can multiply a NumPy matrix of float64 values by the builtin in 2 just by writing "m * 2", and it works exactly the way you'd want it to. It's hard to imagine that would be even feasible with an AI-based solution, but with the current design, that's the easiest part of NumPy.)

There are some ideas in your posts that are worth responding to, but I think it's perfectly fair for Guido to decide it's not worth digging through the mass of ignorance about Python's basic design to find the nuggets that can be rejected for more specific reasons instead of just dismissed.

3:53 a.m.

On Thursday, March 6, 2014 8:53:35 PM UTC-6, Andrew Barnert wrote: > >

The only solution without changing Python is to train end-users to write something correct, like Decimal('.1').

```
hi Andrew, yes, that is the problem. And, to be fair, that example
```

is not really the worst because it might be expected that the user should know how to construct Decimals, and could be educated to construct them properly--> Decimal('0.1'); I concur. It is worse in these two scenarios (and others too): sqrt(.1) sin(.1) No one would expect that the user should know to quote the number---when is that ever done? QED this is broken. Again, we know perfectly well why its happening (I am not ignorant) but its not right.

>

The obvious solution for changing Python is to make it easier to create Decimal numbers correctly and/or harder to create them incorrectly. For example, a decimal suffix, as already proposed before this thread, would completely solve the problem:

```
>>> a = 1d
>>> b = .1d
>>> a+b
1.1d
```

```
Yes, and at a bare minimum, that is the immediate change I have been
```

asking for, for now; nothing more.

```
I answered Guido's questions in hopes that he might be willing to
```

dialogue --- not dismiss.

You proposed that Python should handle numbers in an OO way, with numbers being real objects, instances of classes, with a hierarchy including abstract base classes; all of this is already there in Python.

```
Yes, it is... but its not designed to use decimal floating point by
```

default... in order to do that the entire OO setup will have to be changed (what Guido called a sweeping reform). Its like if we want to use floating point decimals in the 21st century, using python, we have to duck tape modules on and then educate users in the correct input of numbers. Seems convoluted to me.

>

You went off on a long digression about how you could implement this using the details of C++-style inheritance, when Python has a completely (and more powerful) different solution to inheritance that has already been used to solve this problem.

```
No, I did not. I answered Guido's questions regarding context as
```

clearly as I could. If python has a more powerful way to handle this situation, gladly do it! I will be happy as a clam to beta test or help with the coding.

>

You proposed some complicated AI-based solution to solve the problem of using separate number classes in a single expression, even though Python (almost exactly like C++, in this case) has already solved that problem with operator overloading.

```
No, I did not. I suggested that unifying numbers in an (AI) way
```

could solve this problem (conceptually) by
regarding all numbers as *PythonNumbers. *Decimals should not only be
default, they should be integrated, not
tacked on with duck tape.

>

(And note that Python is flexible enough that third-party libraries can easily insert new types like quaternions, matrices, symbolic expressions, etc. into the hierarchy in a way that's transparent to end users. I can multiply a NumPy matrix of float64 values by the builtin in 2 just by writing "m * 2", and it works exactly the way you'd want it to. It's hard to imagine that would be even feasible with an AI-based solution, but with the current design, that's the easiest part of NumPy.)

```
That's nice for you. Because sqrt(.23709) does not behave as I
```

expect, sadly, I have to train my users to enter sqrt('0.23709').

>

There are some ideas in your posts that are worth responding to,

```
Thank you. If a user goes to the time and trouble to present an
```

idea clearly, I would expect the responders to respect the effort and respond to the points that make sense.

```
Andrew, I respect you for taking the time to dialogue, I appreciate
```

it. Thanks.

marcus

4:32 a.m.

On 06Mar2014 19:53, Mark H. Harris harrismh777@gmail.com wrote:

On Thursday, March 6, 2014 8:53:35 PM UTC-6, Andrew Barnert wrote: [... lots of stuff, snipped along with Mark's replies ...]

You proposed some complicated AI-based solution to solve the problem of using separate number classes in a single expression, even though Python (almost exactly like C++, in this case) has already solved that problem with operator overloading.

```
No, I did not. I suggested that unifying numbers in an (AI) way
```

could solve this problem (conceptually) by
regarding all numbers as *PythonNumbers. *Decimals should not only be
default, they should be integrated, not
tacked on with duck tape.

This sounds to me like keeping the original numeric text around and only turning it into some efficient-for-a-machine representation when it comes time to work on it.

One issue with that is that when the work occurs, it can be far from where the number was defined, and the correct internal representation might be poorly chosen.

(And note that Python is flexible enough that third-party libraries can easily insert new types like quaternions, matrices, symbolic expressions, etc. into the hierarchy in a way that's transparent to end users. I can multiply a NumPy matrix of float64 values by the builtin in 2 just by writing "m * 2", and it works exactly the way you'd want it to. It's hard to imagine that would be even feasible with an AI-based solution, but with the current design, that's the easiest part of NumPy.)

```
That's nice for you. Because sqrt(.23709) does not behave as I
```

expect, sadly, I have to train my users to enter sqrt('0.23709').

That's partly because .23709, in current python, becomes an IEEE float with some loss of precision because binary fractions and base-10 fractions do not round trip.

But also partly because sqrt() (in the pure mathematical sense) often produces transcendental numbers, which are not representation by any fixed precision intergal base notation - effectively IEEE floats and Decimal floats are fractions/rationals.

So sqrt() will, for almost all numbers, involve loss of precision no matter what base your backing storage is: base 2 as in IEEE float or base 10 as in a Decimal. Steven (or Stephen) has already pointed this out to you; perhaps it has been missed.

And going back (eg sqrt(2.01)*sqrt(2.01) ==> not-quite-2.01) just extends this loss of precision.

This is an inherent problem unless sqrt() doesn't convert to a "concrete" type, and instead just lurks as some symbolic representation, so that any expression involving it becomes a progressively more elaboration representation of an algebraic expression. Which may be correct, but only until you try to evaluate it to get a concrete number again.

There are some ideas in your posts that are worth responding to,

```
Thank you. If a user goes to the time and trouble to present an
```

idea clearly, I would expect the responders to respect the effort and respond to the points that make sense.

Many people have. You should see the short shrift some ideas receive.

I think part of the problem is not your wish for a "number" but that lack of a concrete proposal to implement it.

Cameron Simpson cs@zip.com.au

The Usenet is not the real world. The Usenet usually does not even resemble the real world. - spaf@cs.purdue.edu

5:30 a.m.

On Mar 6, 2014, at 19:53, "Mark H. Harris" harrismh777@gmail.com wrote:

On Thursday, March 6, 2014 8:53:35 PM UTC-6, Andrew Barnert wrote:

The only solution without changing Python is to train end-users to write something correct, like Decimal('.1').

```
hi Andrew, yes, that is the problem. And, to be fair, that example
is not really the worst
```

because it might be expected that the user should know how to construct Decimals, and could be educated to construct them properly--> Decimal('0.1'); I concur. It is worse in these two scenarios (and others too): sqrt(.1) sin(.1)

No, that's not the same problem but worse, it's a completely different problem.

In the first case, the user is trying to specify 0.1 as a decimal number, which is exactly representable, just not the way he's entered it.

In these cases, the numbers are irrational, and therefore inherently impossible to represent exactly. It doesn't matter whether you use decimal floats or binary floats.

```
No one would expect that the user should
know to quote the number---when is that ever done?
```

What does quoting have to do with anything? Do you not understand why Decimal('.1') works? It's not because Python is a weakly-typed language that allows you to use strings as numbers, but because Decimal has a constructor that takes strings, for a specific and well-documented reason. Quoting here would just give you a TypeError. And nothing in your proposal(s) would change that.

```
QED this is broken. Again, we know
perfectly well why its happening (I am not ignorant) but its not right.
```

The only way to fix this second problem is to use a symbolic representation instead of a numeric one.

The good news is that Python's design makes that pretty easy to add on--as SymPy demonstrates. Your proposal would actually make this kind of add-on much harder.

The obvious
solution for changing Python is to make it easier to create Decimal numbers correctly
and/or harder to create them incorrectly.

Yes, and at a bare minimum, that is the immediate change I have been asking for,
for now; nothing more.

And people have agreed with that, and proposed feasible extensions to it. Presenting it as the first step toward some radical and ill-formed transformation of the whole language weakens the case for this suggestion. Implying that it would solve problems (like handling irrational numbers) that it obviously can't also weakens the case.

```
I answered Guido's questions in hopes that
he might be willing to dialogue --- not dismiss.
```

He was willing to dialogue, as evidenced by his initial replies. It was only after you demonstrated your ignorance of basics fundamentals of Python (as a user, not even about its implementation) and math/numerics, and implied that you had no interest in correcting that ignorance, that he dismissed you. And he has every right to do so. He's the one donating his free time to make a great language for you (and many others) to use, not the other way around.

You proposed that Python should handle numbers in an OO way, with numbers being real objects, instances of classes, with a hierarchy including abstract base classes; all of this is already there in Python.

```
Yes, it is... but its not designed to use decimal floating point by
default... in order to do that the entire OO
```

setup will have to be changed (what Guido called a sweeping reform).

Nonsense. Python already has classes for both binary and decimal floats. They both fit into the hierarchy properly. They both interact with other types the way they should. Changing which one you get from the literal "0.1" would be a simple change to the parser, and have no effect whatsoever to the OO setup.

You went off
on a long digression about how you could implement this using the details of C++-style
inheritance, when Python has a completely (and more powerful) different solution to
inheritance that has already been used to solve this problem.

No, I did not. I answered Guido's questions regarding context as clearly as I
could. If python has a more powerful
way to handle this situation, gladly do it! I will be happy as a clam to beta test or
help with the coding.

It's already been done, years ago, so nobody has to do it. There's already an OO system that works, with abstract and concrete classes, and with a rich operator overloading system. And it's already been used to give you all of the abstract and concrete numeric types you want, and they do the things you asked for in this section, all without having to do any jumps through virtual pointer tables.

```
No, I did not. I suggested that unifying numbers in an (AI) way
could solve this problem (conceptually) by
```

regarding all numbers as PythonNumbers.

So you didn't suggest a complicated AI-based solution, you suggested a complicated AI-based solution?

Decimals should not only be default, they should be integrated, not tacked on with duck tape.

How does that have anything to do with the first half of this paragraph?

And in what way are Decimals "tacked on with duck tape"? They're instances of Number, and of Real. They act like numbers of other types, including interacting properly with other types like int. What is missing from the Decimal type and the ABCs in Number that makes you think we need a radical change?

If all you're suggesting is moving Decimal from the decimal module to builtins and/or adding parser support for decimal literals, those are not sweeping changes, they're both very simple changes. (That doesn't necessarily mean they're _desirable_ changes, but that's another argument--an argument nobody can actually begin until you make it clear whether or not that's what you're suggesting, which can't happen until you learn enough about using Python to know what you're suggesting.)

```
That's nice for you. Because sqrt(.23709) does not behave as I
expect, sadly, I have to train my users to enter sqrt('0.23709').
```

How do you expect it to behave? Again, using decimal floats here would make no difference. Instead of needing an infinite number of binary digits, you need an infinite number of decimal digits. Dividing infinite by log2(10) still leaves infinity. If you're trying to fix that, you're not trying to fix Python, you're trying to fix irrational numbers. This list cannot help you with that. Prayer is the only option, but medieval mathematicians tried that and didn't get very far, and eventually we had to accept that there are numbers that cannot be represented finitely.

There are some ideas in your posts that are worth responding to,

```
Thank you. If a user goes to the time and trouble to present an idea
clearly, I would expect the responders
```

to respect the effort and respond to the points that make sense.

```
Andrew, I respect you for taking the time to dialogue, I appreciate it.
Thanks.
```

marcus

6:01 a.m.

On Thursday, March 6, 2014 11:30:47 PM UTC-6, Andrew Barnert wrote: >

```
No one would expect that the user should
know to quote the
```

number---when is that ever done?

What does quoting have to do with anything? Do you not understand why Decimal('.1') works? It's not because Python is a weakly-typed language that allows you to use strings as numbers, but because Decimal has a constructor that takes strings, for a specific and well-documented reason. Quoting here would just give you a TypeError. And nothing in your proposal(s) would change that.

I understand precisely what it happening. It is just the opposite of what you are implying (that I'm ignorant) which you actually state later. I full well know that the quoted string is forcing the right constructor to be called for the Decimal object. I am an experienced OO programmer and have many years under my belt with C++, Smalltalk, and Java. Please don't be pedantic with me, nor imply that I am ignorant. It does not become you, and it irritates me... besides, it gets in the way of discussion. My point is not about loose typing, nor about Decimal constructors, ... my point is that an average user entering a number into a sqrt() function is not going to realize that they need to enter a quoted string in order to force the string constructor on the Decimal Class !! They are going to want to enter this --> sqrt( .789 ) not this sqrt( '0.789' )

I know how it works. I'm asking for a reasonable response to the need for normal usability.

7:20 p.m.

Mark,

I don't know if it has occurred to you, but several of the rhetorical forms you have used are pretty upsetting to many Python devs. (I can give you a list offline.) It would really help the discussion if you tried to keep your rhetoric in check. (A few other participants might also want to focus more on the topic and less on Mark.)

Now I'd like to restate the core issue in ways that should appeal to Python devs. I am describing the status quo in Python 3.4, since that's the starting point for any evolution of Python. Please bear with me as I describe the status quo -- I find it necessary as a context for any future proposals.

Python's parser recognizes a few different types of number literals: integers (e.g. 42), "decimal point" notation (e.g. 3.14), "exponential notation" (e.g. 2.1e12), and any of these followed by 'j'. There are also binary, octal and hex notations for integers (e.g. 0b1010, 0o12, 0xa). The parser, by design, does not have any understanding of the context in which these numbers are used -- just as it doesn't know about the types of variables (even when it's obvious -- e.g. after seeing "x = 4" the parser does not keep track of the fact that x is now an integer with the value 4, although it does remember it has seen an assignment to x).

What the parser does with those number literals (as with string literals and with the keywords None, True, False) is that it creates an object of an appropriate type with an appropriate value. This object is then used as a value in the expression in which it occurs.

The current algorithm for creating the object is to create an int (which is
implemented as an exactly represented arbitrary-precision integer) when the
literal is either binary/octal/hex or decimal without point or exponent, a
Python float (which is an object wrapping an IEEE 754 double) when a
decimal point or exponent is present, and a Python complex (which wraps
*two* IEEE doubles) when a trailing 'j' is seen.

Note that negative numbers are not literals -- e.g. -10 is an expression which applies the unary minus operator to an integer object with the value

- (Similarly, IEEE inf and NaN don't have literals but requires evaluating an expression, e.g. float('inf').)

Not all numbers come from literals -- most come from expressions such as x+y, some are read from files or other input media. (E.g. the struct module can "read" IEEE floats and doubles from byte strings containing their "native" binary encoding, and float() with a string argument can interpret decimal numbers with optional decimal point and/or exponent.)

The clever thing about expressions (in Python as in many languages) is that the overloading of operators can make the "right" think happen when numbers of different types are combined. Consider x+y. If x and y are both ints, the result is an int (and is computed exactly). If either is a float and the other an int, the result is a float. If either is a complex and the other is an int or float, the result is a float.

If you are defining your own number-ish type in Python or C code, you can
make it play this game too, and this is how the current Decimal and
Fraction types work. Python's operator overloading machinery is flexible
enough so that e.g. Fraction can say "if a Fraction and an int are added,
the result is a Fraction; but if a Fraction and a float are added, the
result is a float." (Because Python is dynamic, it would be *possible* to
define a Fraction type that returns a plain int for results whose
denominator is 1, but the stdlib's Fraction type doesn't do this.)

Most of the time, the built-in types are conservative in their return type
(by which I mean that adding two ints returns an int, not a float), but
there are a few exceptions: int/int returns a float (because returning an
int would require truncation or a compound (div, mod) result), and the **
(power) operator also has a few special cases (int**negative_int and
negative_float**non_integer).

Now let's move on to the problems, which begin with the current float type. Given the representation it is obvious that various limitations exist; results may have to be truncated to fit into 64 bits, and conversion to and from decimal cannot always preserve the exact value. It turns out that the truncation of results doesn't bother most users (everyone knows what a typical hand calculator does for 1/3), but the issues around conversion to and from decimal often trip over users on their initial forays into learning the language. The improvement we made to the output conversion (dropping digits that don't affect round-tripping) take some of the sting out of this, but Python newbies still excitedly point out some of the unavoidable anomalies like 1.1 + 2.2 ==> 3.3000000000000003 when they first discover it.

So what to do about it?

If we aren't too worried about strict backwards compatibility, there's an easy answer that should shut up the newbies: change the parser so that when it sees a number with a decimal point or an exponent it returns a Decimal instance, and make a bunch of other adjustments to match. (E.g repr() of the Decimal 3.14 should return '3.14' instead of "Decimal('3.13')".) The float() builtin should also return a Decimal (we should probably just rename the whole type to 'float'). The math module will have to be rewritten. We should probably preserve the existing binary float under some other name, perhaps requiring an import. We'll have to decide what to do about complex numbers. (One option would be to remove supporting them in the parser.) The details can be worked out in a PEP. I don't expect this process to be easy or without controversies, but I'm confident we could come up with a good design.

But there are big problems with such a proposal.

Those newbies perhaps aren't Python's most important user population. (And for many of them it is actually a moment of enlightenment on their way to becoming experts, once they hear the explanation.) The problems with binary floats don't affect most actual calculations (never mind 1.1+2.2, once you've seen what 1/3 looks like you switch all your output to some kind of rounded format). Many of the problems with floating point that actually matter (such as truncation, or infinities, or NaN) don't go away just by switching everything to Decimal.

For many Python programs the switch to decimal would not matter at all. For example, in all my own recent coding (which focuses on network I/O), the only uses I've had for float is for timekeeping (which is inherently approximate) and to prints some percentages in a report. Such code wouldn't be affected at all -- it wouldn't break, but it wouldn't be any simpler either. Decimal/binary simply isn't an issue here.

However, for the growing contingent of scientists who use Python as a replacement for Matlab (not Mathematica!), it could be a big nuisance. They don't care about decimal issues (they actually like binary better) and they write lots of C++ code that interfaces between CPython's internal API and various C++ libraries for numeric data processing, all of which use IEEE binary. (If anything, they'd probably want a 32-bit binary float literal more than a decimal float literal.) Python already defines some macros for converting between standard Python float objects and C++ doubles (e.g. PyFloat_AS_DOUBLE), so technically we could support source-code compatibility for these users, but performance would definitely suffer (currently that macro just returns the C++ double value already present in the object; it would have to be changed to call a costly decimal-to-binary macro).

There is also the huge effort of actually implementing such a proposal. It's not insurmountable, but I estimate it would be at least as much work as the str->unicode conversion we did for Python 3.

All in all I just don't see the stars aligned for this proposal. (Nor for anything even more sweeping like representing 1/3 as a fraction or some other symbolic representation.)

So what to do instead? A few simper proposals have been made.

Maybe we can add a new literal notation meaning 'decimal'; that would be a relatively straightforward change to the parser (once the new decimal extension module is incorporated), but it would not do much to avoid surprising newbies (certainly you can't go teaching them to always write 3.14d instead of 3.14). However, it would probably help out frequent users of Decimal. (Then again, they might not be using literals that much -- I imagine the source of most such programs is user or file input.)

I'm not excited about a notation for Fraction -- the use cases are too esoteric.

Maybe we can fix the conversion between Decimal and float (if this is really all that matters to Mark, as it appears to be from his last email -- I'd already written most of the above before it arrived). Could it be as simple as converting the float to a string using repr()? Given the smarts in the float repr() that should fix the examples Mark complained about. Are there any roadblocks in the implementation or assumptions of the Decimal type here? Perhaps the default Decimal context being set for a much higher precision makes it philosophically unacceptable?

I think I've said as much as I can, for now.

-- --Guido van Rossum (python.org/~guido)

11:42 p.m.

On 7 March 2014 19:20, Guido van Rossum guido@python.org wrote: >

Maybe we can fix the conversion between Decimal and float (if this is really all that matters to Mark, as it appears to be from his last email -- I'd already written most of the above before it arrived). Could it be as simple as converting the float to a string using repr()? Given the smarts in the float repr() that should fix the examples Mark complained about. Are there any roadblocks in the implementation or assumptions of the Decimal type here?

The current Decimal constructor comes with a guarantee of exactness. It accepts int, str and float (and tuple) and creates a Decimal object having the exact value of the int or float or the exact value that results from a decimal interpretation of the str. This exactness is not mandated by any of the standards on which the decimal module is based but I think it is a good thing. I strongly oppose a change that would have it use heuristics for interpreting other numeric types.

Perhaps the default Decimal context being set for a much higher precision makes it philosophically unacceptable?

The context is ignored by the Decimal constructor which will always create an exact value. If you want the created value rounded to context use unary +:

from decimal import Decimal as D D(0.1) Decimal('0.1000000000000000055511151231257827021181583404541015625') +D(0.1) Decimal('0.1000000000000000055511151231')

Oscar

8 Mar
8 Mar

12:10 a.m.

On Fri, Mar 7, 2014 at 3:42 PM, Oscar Benjamin

oscar.j.benjamin@gmail.comwrote:

On 7 March 2014 19:20, Guido van Rossum guido@python.org wrote: >

Maybe we can fix the conversion between Decimal and float (if this is really all that matters to Mark, as it appears to be from his last email -- I'd already written most of the above before it arrived). Could it be as simple as converting the float to a string using repr()? Given the smarts in the float repr() that should fix the examples Mark complained about. Are there any roadblocks in the implementation or assumptions of the Decimal type here?

The current Decimal constructor comes with a guarantee of exactness.

But Decimal(<float>) is relatively new (it was an error in 2.6). I know
it's annoying to change it again, but I also have the feeling that there's
not much use in producing a Decimal with 53 (or more?) *decimal digits* of
precision from a float with 53 *bits* -- at least not by default. Maybe we
can relent on this guarantee and make the constructor by default use fewer
digits (e.g. using the same algorithm used by repr(<float>)) and have an
optional argument to produce the full precision? Or reserve
Decimal.from_float() for that?

It accepts int, str and float (and tuple) and creates a Decimal object having the exact value of the int or float or the exact value that results from a decimal interpretation of the str. This exactness is not mandated by any of the standards on which the decimal module is based but I think it is a good thing. I strongly oppose a change that would have it use heuristics for interpreting other numeric types.

Well, given that most floating point values are the result of operations
that inherently rounded to 53 bits anyway, I'm not sure that this guarantee
is useful. Sure, I agree that there should be *a* way to construct the
exact Decimal for any IEEE 754 double (since it is always possible, 10
being a multiple of 2), but I'm not sure how important it is that this is
the default constructor.

Perhaps the default Decimal context being set for a much higher precision makes it philosophically unacceptable?

The context is ignored by the Decimal constructor which will always create an exact value. If you want the created value rounded to context use unary +:

from decimal import Decimal as D D(0.1) Decimal('0.1000000000000000055511151231257827021181583404541015625') +D(0.1) Decimal('0.1000000000000000055511151231')

I guess that's too late.

-- --Guido van Rossum (python.org/~guido)

12:27 a.m.

On 8 March 2014 00:10, Guido van Rossum guido@python.org wrote:

On Fri, Mar 7, 2014 at 3:42 PM, Oscar Benjamin oscar.j.benjamin@gmail.com wrote: >

The current Decimal constructor comes with a guarantee of exactness.

But Decimal(<float>) is relatively new (it was an error in 2.6). I know
it's
annoying to change it again, but I also have the feeling that there's not
much use in producing a Decimal with 53 (or more?) *decimal digits* of
precision from a float with 53 *bits* -- at least not by default. Maybe we
can relent on this guarantee and make the constructor by default use fewer
digits (e.g. using the same algorithm used by repr(<float>)) and have an
optional argument to produce the full precision? Or reserve
Decimal.from_float() for that?

I'd rather a TypeError than an inexact conversion. The current behaviour allows to control how rounding occurs. (I don't use the decimal module unless this kind of thing is important).

It accepts int, str and float (and tuple) and creates a Decimal object having the exact value of the int or float or the exact value that results from a decimal interpretation of the str. This exactness is not mandated by any of the standards on which the decimal module is based but I think it is a good thing. I strongly oppose a change that would have it use heuristics for interpreting other numeric types.

Well, given that most floating point values are the result of operations
that inherently rounded to 53 bits anyway, I'm not sure that this guarantee
is useful. Sure, I agree that there should be *a* way to construct the exact
Decimal for any IEEE 754 double (since it is always possible, 10 being a
multiple of 2), but I'm not sure how important it is that this is the
default constructor.

It's hard to reason about the accuracy of calculations that use the decimal module. It's harder than with a fixed-width type because of the possible presence of higher-than-context precision Decimals. The invariant that the constructor is exact (or a TypeError) is of significant benefit when using Decimal for code that accepts inputs of different numeric types.

Oscar

1:02 a.m.

On Fri, Mar 7, 2014 at 4:27 PM, Oscar Benjamin

oscar.j.benjamin@gmail.comwrote:

On 8 March 2014 00:10, Guido van Rossum guido@python.org wrote:

On Fri, Mar 7, 2014 at 3:42 PM, Oscar Benjamin oscar.j.benjamin@gmail.com wrote: >

The current Decimal constructor comes with a guarantee of exactness.

But Decimal(<float>) is relatively new (it was an error in 2.6). I know
it's
annoying to change it again, but I also have the feeling that there's not
much use in producing a Decimal with 53 (or more?) *decimal digits* of
precision from a float with 53 *bits* -- at least not by default. Maybe
we
can relent on this guarantee and make the constructor by default use
fewer
digits (e.g. using the same algorithm used by repr(<float>)) and have an
optional argument to produce the full precision? Or reserve
Decimal.from_float() for that?

I'd rather a TypeError than an inexact conversion. The current behaviour allows to control how rounding occurs. (I don't use the decimal module unless this kind of thing is important).

My proposal still gives you control (e.g. through a keyword argument, or by using Decimal.from_float() instead of the constructor). It just changes the default conversion.

I have to admit I don't recall the discussion that led to accepting float arguments to Decimal(), so I don't know if the exactness guarantee was ever challenged or even discussed. For all other argument types (str, tuple of digits, Decimal) it makes intuitive sense to guarantee exactness, because the input is already in decimal form, so why would the caller pass in digits they don't care about. But for a float the caller has no such control -- I can call round(math.pi, 2) but the resulting float (taken at face value) is still equal to 3.140000000000000124344978758017532527446746826171875, even though just '3.14' is enough as input to float() to produce that exact value.

And unfortunately the default precision for Decimal is much larger than for IEEE double, so using unary + does not get rid of those extraneous digits -- it produces Decimal('3.140000000000000124344978758'). So you really have to work hard to recover produce the intended Decimal('3.14') (or Decimal('3.140000000000000')).

It accepts int, str and float (and tuple) and creates a Decimal object

having the exact value of the int or float or the exact value that results from a decimal interpretation of the str. This exactness is not mandated by any of the standards on which the decimal module is based but I think it is a good thing. I strongly oppose a change that would have it use heuristics for interpreting other numeric types.

Well, given that most floating point values are the result of operations
that inherently rounded to 53 bits anyway, I'm not sure that this
guarantee
is useful. Sure, I agree that there should be *a* way to construct the
exact
Decimal for any IEEE 754 double (since it is always possible, 10 being a
multiple of 2), but I'm not sure how important it is that this is the
default constructor.

It's hard to reason about the accuracy of calculations that use the decimal module. It's harder than with a fixed-width type because of the possible presence of higher-than-context precision Decimals. The invariant that the constructor is exact (or a TypeError) is of significant benefit when using Decimal for code that accepts inputs of different numeric types.

Can you provide an example of this benefit? I can't seem to guess whether you are talking about passing Decimal instances into code that then just uses +, * etc., or whether you are talking about code that uses Decimal internally and takes arguments of different types (e.g. int, float, string).

-- --Guido van Rossum (python.org/~guido)

1:36 a.m.

On Fri, Mar 07, 2014 at 05:02:15PM -0800, Guido van Rossum wrote:

On Fri, Mar 7, 2014 at 4:27 PM, Oscar Benjamin

oscar.j.benjamin@gmail.comwrote:

On 8 March 2014 00:10, Guido van Rossum guido@python.org wrote: [...]

I also have the feeling that there's not
much use in producing a Decimal with 53 (or more?) *decimal digits* of
precision from a float with 53 *bits* -- at least not by default. Maybe
we can relent on this guarantee and make the constructor by default use
fewer digits (e.g. using the same algorithm used by repr(<float>)) and
have an optional argument to produce the full precision? Or reserve
Decimal.from_float() for that?

I'd rather a TypeError than an inexact conversion. The current behaviour allows to control how rounding occurs. (I don't use the decimal module unless this kind of thing is important).

Like Oscar, I would not like to see this change.

My proposal still gives you control (e.g. through a keyword argument, or by using Decimal.from_float() instead of the constructor). It just changes the default conversion.

Who is this default conversion meant to benefit?

I think that anyone passing floats to Decimal ought to know what they are doing, and be quite capable of controlling the rounding via the decimal context, or via an intermediate string:

value = Decimal("%.3f" % some_float)

I believe that Mark wants to use Python as an interactive calculator, and wants decimal-by-default semantics without having to type quotes around his decimal numbers. I suspect that for the amount of time and effort put into this discussion, he probably could have written an interactive calculator using Decimals with the cmd module in half the time :-)

It seems to me that Decimal should not try to guess what precision the user wants for any particular number. Mark is fond of the example of 2.01, and wants to see precisely 2.01, but that assumes that some human being typed in those three digits. If it's the result of some intermediate calculation, there is no reason to believe that a decimal with the value 2.01 exactly is better than one with the value 2.00999999999999978...

Note that the issue is more than just Decimal:

py> from fractions import Fraction py> Fraction(2.01) Fraction(1131529406376837, 562949953421312)

Even more so than for Decimal, I will go to the barricades to defend this exact conversion. Although perhaps Fraction could grow an extra argument to limit the denominator as a short-cut for this:

py> Fraction(2.01).limit_denominator(10000) Fraction(201, 100)

I think it is important that Decimal and Fraction perform conversions as exactly as possible by default. You can always throw precision away afterwards, but you can't recover what was never saved in the first place.

[...]

And unfortunately the default precision for Decimal is much larger than for IEEE double, so using unary + does not get rid of those extraneous digits -- it produces Decimal('3.140000000000000124344978758'). So you really have to work hard to recover produce the intended Decimal('3.14') (or Decimal('3.140000000000000')).

I wouldn't object to Decimal gaining a keyword argument for precision, although it is easy to use an intermediate string. In other words,

```
Decimal(math.pi, prec=2)
```

could be the same as

```
Decimal("%.2f" % math.pi)
```

But what would prec do for non-float arguments? Perhaps we should leave Decimal() alone and put any extra bells and whistles that apply only to floats into the from_float() method.

-- Steven

2:15 a.m.

On Fri, Mar 7, 2014 at 5:36 PM, Steven D'Aprano steve@pearwood.info wrote:

On Fri, Mar 07, 2014 at 05:02:15PM -0800, Guido van Rossum wrote:

On Fri, Mar 7, 2014 at 4:27 PM, Oscar Benjamin

oscar.j.benjamin@gmail.comwrote:

On 8 March 2014 00:10, Guido van Rossum guido@python.org wrote: [...]

I also have the feeling that there's not
much use in producing a Decimal with 53 (or more?) *decimal digits*
of
precision from a float with 53 *bits* -- at least not by default.
Maybe
we can relent on this guarantee and make the constructor by default
use
fewer digits (e.g. using the same algorithm used by repr(<float>))
and
have an optional argument to produce the full precision? Or reserve
Decimal.from_float() for that?

Like Oscar, I would not like to see this change.

Can you argue your position?

My proposal still gives you control (e.g. through a keyword argument, or by using Decimal.from_float() instead of the constructor). It just changes the default conversion.

Who is this default conversion meant to benefit?

I think that anyone passing floats to Decimal ought to know what they are doing,

I think that is provably not the case. (That they don't know -- we can still argue about whether they ought to know.)

and be quite capable of controlling the rounding via the decimal context, or via an intermediate string:

value = Decimal("%.3f" % some_float)

Or, Decimal(repr(some_float)), which DWIM.

I believe that Mark wants to use Python as an interactive calculator, and wants decimal-by-default semantics without having to type quotes around his decimal numbers. I suspect that for the amount of time and effort put into this discussion, he probably could have written an interactive calculator using Decimals with the cmd module in half the time :-)

Mark doesn't like it when we try to guess what he means or believes, but my guess is that he's actually trying to argue on behalf of Python users who would benefit from using Decimal but don't want to be bothered with a lot of rules. Telling such people "use the Decimal class" is much easier than telling them "use the Decimal class, and always put quotes around your numbers" (plus the latter advice is still very approximate, so the real version would be even longer and more confusing).

It seems to me that Decimal should not try to guess what precision the user wants for any particular number. Mark is fond of the example of 2.01, and wants to see precisely 2.01, but that assumes that some human being typed in those three digits. If it's the result of some intermediate calculation, there is no reason to believe that a decimal with the value 2.01 exactly is better than one with the value 2.00999999999999978...

Given the effort that went into making repr(<float>) do the right thing I think that you're dismissing a strawman here. The real proposal does not require guessing intent -- it just uses an existing, superior conversion from float to decimal that is already an integral part of the language. And it uses some static information that Decimal currently completely ignores, i.e. all Python floats (being IEEE doubles) are created with 53 bits of precision.

Note that the issue is more than just Decimal:

py> from fractions import Fraction py> Fraction(2.01) Fraction(1131529406376837, 562949953421312)

Even more so than for Decimal, I will go to the barricades to defend this exact conversion. Although perhaps Fraction could grow an extra argument to limit the denominator as a short-cut for this:

py> Fraction(2.01).limit_denominator(10000) Fraction(201, 100)

An inferior solution. Anyway, nobody cares about Fraction.

I think it is important that Decimal and Fraction perform conversions as exactly as possible by default.

Again, why produce 56 *digits* of precision by default for something that
only intrinsically has 53 *bits*?

You can always throw precision away afterwards, but you can't recover what was never saved in the first place.

There should definitely be *some* API to convert a float to the exact
mathematical Decimal. But I don't think it necessarily needs to be the
default constructor. (It would be different if Decimal was a fixed-point
type. But it isn't.)

[...]

And unfortunately the default precision for Decimal is much larger than for IEEE double, so using unary + does not get rid of those extraneous digits -- it produces Decimal('3.140000000000000124344978758'). So you really have to work hard to recover produce the intended Decimal('3.14') (or Decimal('3.140000000000000')).

I wouldn't object to Decimal gaining a keyword argument for precision, although it is easy to use an intermediate string. In other words,

```
Decimal(math.pi, prec=2)
```

could be the same as

```
Decimal("%.2f" % math.pi)
```

This is actually another strawman. Nobody in this thread has asked for a
way to truncate significant digits like that. It's the *insignificant*
digits that bother some. The question is, what should Decimal(math.pi) do.
It currently returns
Decimal('3.141592653589793115997963468544185161590576171875'), which helps
noone except people interested in how floats are represented internally.

But what would prec do for non-float arguments? Perhaps we should leave Decimal() alone and put any extra bells and whistles that apply only to floats into the from_float() method.

-- --Guido van Rossum (python.org/~guido)

3:55 a.m.

On Fri, Mar 7, 2014 at 6:15 PM, Guido van Rossum guido@python.org wrote:

Or, Decimal(repr(some_float)), which DWIM.

Because I haven't seen anyone else bring this up, using a non-exact conversion breaks the invariant "Decimal(f)==f".

There are so many pairs of numeric types that break that invariant that it might not be a big deal; but in all other cases the invariant is broken because it is theoretically impossible.

from decimal import Decimal f = 2.2 print(Decimal(f)==f) print(Decimal(repr(f))==f)

p

4 a.m.

On Sat, Mar 8, 2014 at 2:55 PM, Paul Du Bois paul.dubois@gmail.com wrote:

On Fri, Mar 7, 2014 at 6:15 PM, Guido van Rossum guido@python.org wrote: >

Or, Decimal(repr(some_float)), which DWIM.

Because I haven't seen anyone else bring this up, using a non-exact conversion breaks the invariant "Decimal(f)==f".

There are so many pairs of numeric types that break that invariant that it might not be a big deal; but in all other cases the invariant is broken because it is theoretically impossible.

from decimal import Decimal f = 2.2 print(Decimal(f)==f) print(Decimal(repr(f))==f)

How is Decimal==float implemented? Is it by calling Decimal(rhs) and then comparing? If so, changing how Decimal(float) works won't break the invariant, as it'll make the same conversion each time.

ChrisA

4:11 a.m.

On Fri, Mar 7, 2014 at 8:00 PM, Chris Angelico rosuav@gmail.com wrote:

On Sat, Mar 8, 2014 at 2:55 PM, Paul Du Bois paul.dubois@gmail.com wrote:

On Fri, Mar 7, 2014 at 6:15 PM, Guido van Rossum guido@python.org wrote: >

Or, Decimal(repr(some_float)), which DWIM.

Because I haven't seen anyone else bring this up, using a non-exact conversion breaks the invariant "Decimal(f)==f".

There are so many pairs of numeric types that break that invariant that it might not be a big deal; but in all other cases the invariant is broken because it is theoretically impossible.

from decimal import Decimal f = 2.2 print(Decimal(f)==f) print(Decimal(repr(f))==f)

How is Decimal==float implemented? Is it by calling Decimal(rhs) and then comparing? If so, changing how Decimal(float) works won't break the invariant, as it'll make the same conversion each time.

There's a special function to convert the other operand for the purpose of comparisons, and it currently uses Decimal.from_float(). I am not (yet :-) proposing to change the behavior of that function -- it is and has always been the API function that does exact float-to-Decimal conversion. Decimal(repr(f)) == f returns False today (for the majority of float values anyway) and under my proposal Decimal(f) == f would also return False. I don't (yet :-) think that's a deal breaker.

-- --Guido van Rossum (python.org/~guido)

4:16 a.m.

On Sat, Mar 8, 2014 at 3:11 PM, Guido van Rossum guido@python.org wrote:

How is Decimal==float implemented? Is it by calling Decimal(rhs) and then comparing? If so, changing how Decimal(float) works won't break the invariant, as it'll make the same conversion each time.

There's a special function to convert the other operand for the purpose of comparisons, and it currently uses Decimal.from_float(). I am not (yet :-) proposing to change the behavior of that function -- it is and has always been the API function that does exact float-to-Decimal conversion. Decimal(repr(f)) == f returns False today (for the majority of float values anyway) and under my proposal Decimal(f) == f would also return False. I don't (yet :-) think that's a deal breaker.

Maybe not a deal breaker, but certainly another panel on the bikeshed. It would make good sense to retain that invariant by making the comparison use repr just like the constructor.

ChrisA

4:38 a.m.

On Fri, Mar 7, 2014 at 8:16 PM, Chris Angelico rosuav@gmail.com wrote:

On Sat, Mar 8, 2014 at 3:11 PM, Guido van Rossum guido@python.org wrote:

There's a special function to convert the other operand for the purpose of comparisons, and it currently uses Decimal.from_float(). I am not (yet :-) proposing to change the behavior of that function -- it is and has always been the API function that does exact float-to-Decimal conversion. Decimal(repr(f)) == f returns False today (for the majority of float values anyway) and under my proposal Decimal(f) == f would also return False. I don't (yet :-) think that's a deal breaker.

Maybe not a deal breaker, but certainly another panel on the bikeshed. It would make good sense to retain that invariant by making the comparison use repr just like the constructor.

Not sure I agree. I find the invariant Decimal(x) == x rather uninteresting. I find the idea that Decimal.__eq__ compares the exact mathematical values very appealing.

-- --Guido van Rossum (python.org/~guido)

4:40 a.m.

Sometime today I think I wrote that the C implementation of Decimal isn't yet in the stdlib. I stand corrected. It is. Yay!

-- --Guido van Rossum (python.org/~guido)

4:39 a.m.

[Guido]

Decimal(repr(f)) == f returns False today (for the majority of float values anyway) and under my proposal Decimal(f) == f would also return False. I don't (yet :-) think that's a deal breaker.

To the contrary, that's "a feature". Current Pythons take equality testing seriously, striving to return True when & only when values are mathematically ("as if using infinite precision") identical, even when that's inconvenient to do:

2**500 == 2.0**500
True
2**500 + 1 == 2.0**500
False

If information is lost when converting, it's doing nobody a favor for "==" to pretend it hasn't been lost. So long as Decimal.from_float(f) == f is still True, then equality would be working as expected in all cases.

4:44 a.m.

On Fri, Mar 7, 2014 at 8:39 PM, Tim Peters tim.peters@gmail.com wrote:

[Guido]

Decimal(repr(f)) == f returns False today (for the majority of float values anyway) and under my proposal Decimal(f) == f would also return False. I don't (yet :-) think that's a deal breaker.

To the contrary, that's "a feature". Current Pythons take equality testing seriously, striving to return True when & only when values are mathematically ("as if using infinite precision") identical, even when that's inconvenient to do:

2**500 == 2.0**500
True
2**500 + 1 == 2.0**500
False

If information is lost when converting, it's doing nobody a favor for "==" to pretend it hasn't been lost. So long as Decimal.from_float(f) == f is still True, then equality would be working as expected in all cases.

I agree. But what do you think of my main proposal? I would retract it if you advised me so.

-- --Guido van Rossum (python.org/~guido)

5:01 a.m.

[Guido]

I agree. But what do you think of my main proposal? I would retract it if you advised me so.

Oh, I stick to tiny detail these days, to avoid getting sucked into endless pointless arguments ;-)

That said, I agree that the default Decimal constructor acting like Decimal(x) == Decimal(repr(x)) would be far less surprising to the vast majority of users, and no problem at all for "expert" users (provided Decimal.from_float(x) were still available).

Against it, the change would invalidate dozens of stackoverflow answers advising users to try printing Decimal(x) to see where their naive binary floating-point expectations are going wrong. And I have no feel for how much production code may be relying on current Decimal(x) behavior (I have none).

But, if we're willing to risk changing the meaning of bool(datetime,time(arguments creating an object in some timezones - but not all - that converts to midnight UTC )), then I suppose the floodgates are waaaaaay open now ;-)

5:33 a.m.

On Mar 7, 2014, at 20:44, Guido van Rossum guido@python.org wrote:

I agree. But what do you think of my main proposal? I would retract it if you advised me so.

I think this may be a good time to raise my points from off-list, in hopes that someone can say "those potential problems are not real problems anyone would ever care about".

Today, given any real, Decimal(float('real')) always gives you the value of the closest binary float to that real. With this change, it will sometimes give you the value of the second-closest repr-of-a-binary-float to that real. This means the error across any range of reals increases. It's still below the rule-of-thumb cutoff that everyone uses for converting through floats, but it is higher by a nonzero amount that doesn't cancel out.

Similarly, today, the distribution of float values across the real number line is... not uniform (because of exponents), and I don't know the right technical term, you know what it looks like. The distribution of repr-of-float variables is not.

Do users of Decimal (or, rather, users of Decimal(float) conversion) care about either of those issues? I don't know. I've never written any code that converted floats to Decimals and then used them as anything other than low-precision fixed-point values (like dollars and cents).

By the way, to dismiss any potential performance worries: with both 3.3 and trunk, Decimal(repr(f)) is actually faster than Decimal.from_float(f) by a factor of 1.3 to 3.2 in a variety of quick tests. If that weren't true, you might have to define Decimal(f) as if it were building and then parsing a string but them implement more complicated code that feeds digits from one algorithm to the other, but fortunately, it looks like this is a three-line change.

6:36 a.m.

On Fri, Mar 07, 2014 at 09:33:38PM -0800, Andrew Barnert wrote:

Today, given any real, Decimal(float('real')) always gives you the value of the closest binary float to that real. With this change, it will sometimes give you the value of the second-closest repr-of-a-binary-float to that real.

Please don't talk about "reals". "Real" has a technical meaning in mathematics, and no computer number system is able to represent the reals. In Python, we have floats, which are C doubles, and Decimals.

Putting that aside, I'm afraid I don't understand how Guido's suggestion to use repr() in the Decimal constructor will sometimes give the second-closest value. Can you explain in more detail? If you can show an example, that would be great too.

This means the error across any range of reals increases. It's still below the rule-of-thumb cutoff that everyone uses for converting through floats, but it is higher by a nonzero amount that doesn't cancel out.

Similarly, today, the distribution of float values across the real number line is... not uniform (because of exponents), and I don't know the right technical term, you know what it looks like. The distribution of repr-of-float variables is not.

Nope, sorry, I don't get you.

I think that what you are trying to say is that for each binary exponent, the floats with that same exponent are distributed uniformly:

|...|...|...|...|...|...|...|...|

but their reprs are not:

|...|..|..|....|...|....|..|....|

Is this what you mean?

-- Steven

7:53 a.m.

From: Steven D'Aprano steve@pearwood.info

Sent: Friday, March 7, 2014 10:36 PM

On Fri, Mar 07, 2014 at 09:33:38PM -0800, Andrew Barnert wrote:

Today, given any real, Decimal(float('real')) always gives you the value of the closest binary float to that real. With this change, it will sometimes give you the value of the second-closest repr-of-a-binary-float to that real.

Please don't talk about "reals". "Real" has a technical meaning in mathematics, and no computer number system is able to represent the reals. In Python, we have floats, which are C doubles, and Decimals.

I am using "real" in the technical sense. That's the whole crux of the problem: you want to use the real[1] number 0.1 in Python, and you can't, because there is no IEEE double that matches that value. If we ignore reals and only talk about floats, then 0.1 and 0.100000000000000012 are the same number, so there is no problem.

[1]: I suppose we could talk about rationals instead of reals here, because (a) you can't enter irrationals as literals, and (b) (b) Guido's proposal obviously doesn't attempt to help with them. I don't think that makes a difference here, but I could be wrong; if I am, please point out below where this leads me astray.

Put another way: This cannot be just about converting IEEE double floats to Decimal floats, because Python already does that perfectly. It's about recovering information lost in representing real numbers (or, if you prefer, rational numbers, or Python literals) as IEEE double floats, when converting them to Decimal floats, by taking advantage of the fact that (a) we know that humans tend to use numbers like 0.1 more often than numbers like 0.100000000000000005551, and (b) we know the precision of IEEE double and can guarantee that we're not losing any real information.

Putting that aside, I'm afraid I don't understand how Guido's suggestion to use repr() in the Decimal constructor will sometimes give the second-closest value. Can you explain in more detail? If you can show an example, that would be great too.

OK, take the number 0.100000000000000012.

The closest IEEE double to this number is 0.1000000000000000055511151231257827021181583404541015625. The next closest is 0.10000000000000001942890293094023945741355419158935546875. Today's design gives you the former when you write Decimal(0.100000000000000012). You get the closest one in this case, and in every case.

The closest repr of an IEEE double to this number is 0.10000000000000002. But with Guido's proposal, the one you get is 0.1, which is farther from the number you wanted.

So, if you treat evaluate-literal-as-float-then-Decimal.from_float as a (mathematical) function, it has the property that it always gives you the closest value from its range to your input. If you treat evaluate-literal-as-float-then-repr-then-Decimal as a function, it does not have that property.

This means the error across any

range of reals increases. It's still below the rule-of-thumb cutoff that everyone uses for converting through floats, but it is higher by a nonzero amount that doesn't cancel out.

Was this second point clear? In case it wasn't, let me expand on it:

In the example above, Guido's proposal gives you nearly double the error (1.2e-17 vs. 6.4+e-18) of the current design. Of course there are also cases where it gives a lower error, 0.1 being an obvious example. I'm pretty sure (but I don't know how to prove…) that if you integrate the absolute error over any sufficiently large[1] set of reals, it's higher in Guido's proposal than in the current design.

[1]: The "sufficiently large" there is just so you don't choose a set so small that all of its elements map to a single IEEE double, like { 0.1 <= x < 0.100000000000000001 }.

Similarly, today, the distribution of float values across the real

number line is... not uniform (because of exponents), and I don't know the right technical term, you know what it looks like. The distribution of repr-of-float variables is not.

Nope, sorry, I don't get you.

I think that what you are trying to say is that for each binary exponent, the floats with that same exponent are distributed uniformly:

|...|...|...|...|...|...|...|...|

but their reprs are not:

|...|..|..|....|...|....|..|....|

Is this what you mean?

That's part of it. But on top of that, I think (but again can't prove) that the reprs are on average skewed further to the left than the right, which means that, e.g., the average of a collection of reprs-of-floats will also be skewed in that direction.

So, those are my three points. Remember that I don't know if any of them actually matter to anyone, and in fact am hoping they do not.

11:20 a.m.

On 3/7/2014 11:44 PM, Guido van Rossum wrote:

I agree. But what do you think of my main proposal? I would retract it if you advised me so.

I'd like to hear what Mark Dickinson has to say. I assume like most people he lost interest in this thread. I've sent him a message asking him to take a look.

Eric.

3:59 a.m.

On Friday, March 7, 2014 8:15:11 PM UTC-6, Guido van Rossum wrote: > >

. . . but my guess is that he's actually trying to argue on behalf of Python users who would benefit from using Decimal but don't want to be bothered with a lot of rules. Telling such people "use the Decimal class" is much easier than telling them "use the Decimal class, and always put quotes around your numbers" (plus the latter advice is still very approximate, so the real version would be even longer and more confusing).

Yes, that is correct. Although, I would modify "with certain rules". I mean, its ok to expect some rules (I think that's what we mean by semantic). But rules that seem to go against the grain of the normal human experience ... like quoting a numerical input. I mean if you tell me as a programmer that's what I have to do, well that's what I have to do. On the other hand (the hand I'm really concerned about) if you tell a naive user they have to quote their numerical inputs-- that seems like the wrong approach. Yes, its much better to say, "Please use the Decimal module".

marcus

9:43 a.m.

On Sat, Mar 8, 2014 at 12:10 AM, Guido van Rossum guido@python.org wrote:

>

*decimal digits* of
precision from a float with 53 *bits* -- at least not by default. Maybe we
can relent on this guarantee and make the constructor by default use fewer
digits (e.g. using the same algorithm used by repr(<float>)) and have an
optional argument to produce the full precision? Or reserve
Decimal.from_float() for that?

Using the same algorithm as used by repr(float) feels like a bad idea to me: my gut feeling is that a conversion from float to Decimal is a fundamental operation that should be easily describable in simple terms, and should not involve the level of complication and 'magic' involved in the float repr. (That level of magic seems fine for the repr, because in most cases you don't actually care too much what string you get so long as it evaluates back to the correct float; you're not doing arithmetic with the result.)

IEEE 754-2008 requires simply that conversions between numbers in different formats round correctly. It specifies a "convertFormat" operation, described as follows:

""" If the conversion is to a format in a different radix or to a narrower precision in the same radix, the result shall be rounded as specified in Clause 4. Conversion to a format with the same radix but wider precision and range is always exact. ... """

... where clause 4 is the one that outlines the general rounding rules that apply to almost all IEEE 754 operations.

I see three sane options for float to Decimal conversion:

- Raise an exception.
- Round to the nearest Decimal value using the current context for that round operation.
- Do what we're currently doing, and do an exact conversion.

Option 2 would seem closest to what IEEE 754 specifies. To augment option 2, we could also make it easier to create Decimal contexts corresponding to the IEEE 754 recommended interchange formats (decimal32, decimal64, decimal128). This was already proposed in http://bugs.python.org/issue8786.

Mark

10:01 a.m.

On Sat, Mar 8, 2014 at 9:43 AM, Mark Dickinson dickinsm@gmail.com wrote:

>

I see three sane options for float to Decimal conversion:

- Raise an exception.
- Round to the nearest Decimal value using the current context for that round operation.
- Do what we're currently doing, and do an exact conversion.

Proposals for change should also take into consideration that Decimal
already does *exact* conversions for integers (and I believe has done since
it first existed). It would be quite surprising for `Decimal(2**1000)`

and
`Decimal(2.0**1000)`

to be different numbers. If we change the float
behaviour, we might also want to change conversion from int to round to the
nearest Decimal using the current context. Again, that's closer to what
IEEE 754 specifies for the "convertFromInt" operation.

On the other hand, we probably shouldn't lend *too* much weight to IEEE
754, especially when talking about choice of precision. IEEE 754 isn't a
perfect fit for Decimal: the IEEE standard is mostly concerned with fixed
width decimal formats, which is subtly different from Mike Cowlishaw's
approach of "extensible precision" where the precision is not so closely
tied to the format. Python's decimal module is based on Cowlishaw's
standard, not on IEEE 754.

Mark

4:54 p.m.

I'll try to respond to Mark Dickinson's second message (and nothing else that happens in the thread since last night), because (a) it concisely summarizes his position and (b) brings up a new strawman.

On Sat, Mar 8, 2014 at 2:01 AM, Mark Dickinson dickinsm@gmail.com wrote:

On Sat, Mar 8, 2014 at 9:43 AM, Mark Dickinson dickinsm@gmail.com wrote:

>

I see three sane options for float to Decimal conversion:

- Raise an exception.
- Round to the nearest Decimal value using the current context for that round operation.
- Do what we're currently doing, and do an exact conversion.

I think you're writing this entirely from the POV of an expert in floating point. And I'm glad we have experts like you around! I don't consider myself an expert at all, but I do think I have something to bring to the table -- insight the experience of non-expert users.

When a non-expert writes Decimal(1.1), each of the three above outcomes surprises. We know that (1) was unpopular, that's why we changed it. We now know that (3) is unpopular at least in some circles (Mark Harrison can't be the only one who doesn't like it). Changing to (2) wouldn't do much to address this, because the default context has way more precision than float, so it still shows a lot of extraneous digits.

Yes, it's trivial to get rid of those extra digits, just use quotes. But if
*my* proposal were adopted, it would be trivial for numerical experts to
get the extra digits -- just use from_float(). At this point, my claim is
that we're talking essentially about what is the better experience for most
users, and while I am not much of a user of Decimal myself, I believe that
my proposal has benefits more people and situations than it has downsides.

Proposals for change should also take into
consideration that Decimal
already does *exact* conversions for integers (and I believe has done since
it first existed). It would be quite surprising for `Decimal(2**1000)`

and
`Decimal(2.0**1000)`

to be different numbers.

This feels like a strawman, since Decimal(2**1000 + 1) and
Decimal(2.0**1000 + 1) produce different outcomes (the latter gives a lot
of digits but is one too small), and the similar examples with a large
exponent (10000) differ dramatically (the float version raises an
exception).

For most purposes it's a fluke that 2.0**1000 can be represented exactly as a float, and the argument doesn't convince me at all. There are just too many odd examples like that, and it always will remain a minefield.

If we change the float behaviour, we might also want to change conversion from int to round to the nearest Decimal using the current context. Again, that's closer to what IEEE 754 specifies for the "convertFromInt" operation.

I don't think that's no the table. Python's int doesn't lose precision, and
users know this and depend on it. (In fact, I find ignoring the current
context in the constructor a totally reasonable approach -- it does this
uniformly, regardless of the argument type. My proposal doesn't change
this, the context would *still* not be used to decide how a float is
converted. Only the fact that it's a float would.)

>

On the other hand, we probably shouldn't lend
*too* much weight to IEEE
754, especially when talking about choice of precision. IEEE 754 isn't a
perfect fit for Decimal: the IEEE standard is mostly concerned with fixed
width decimal formats, which is subtly different from Mike Cowlishaw's
approach of "extensible precision" where the precision is not so closely
tied to the format. Python's decimal module is based on Cowlishaw's
standard, not on IEEE 754.

I wonder what Cowlishaw would say about our current discussion. He is also the father of Rexx...

-- --Guido van Rossum (python.org/~guido)

6:11 p.m.

On 8 March 2014 16:54, Guido van Rossum guido@python.org wrote:

I'll try to respond to Mark Dickinson's second message (and nothing else that happens in the thread since last night), because (a) it concisely summarizes his position and (b) brings up a new strawman.

On Sat, Mar 8, 2014 at 2:01 AM, Mark Dickinson dickinsm@gmail.com wrote: >

On Sat, Mar 8, 2014 at 9:43 AM, Mark Dickinson dickinsm@gmail.com wrote: > >

I see three sane options for float to Decimal conversion:

- Raise an exception.
- Round to the nearest Decimal value using the current context for that round operation.
- Do what we're currently doing, and do an exact conversion.

I think you're writing this entirely from the POV of an expert in floating point. And I'm glad we have experts like you around! I don't consider myself an expert at all, but I do think I have something to bring to the table -- insight the experience of non-expert users.

Standards compliance is important though. Mark is essentially restating something that is repeated hundreds of times throughout various floating point standards documents: a result may be exact or it must be correctly rounded according to the current context (with appropriate flags set and traps fired).

It is not mandated that results requiring more precision than the current context be exact. It is mandated that if a result is to be inexact then the implementation must use a very specific type of inexactness. I don't believe that a standards-compliant decimal module has any wiggle room to invent a new kind of rounding (which I think would be required to achieve what you suggest).

When a non-expert writes Decimal(1.1), each of the three above outcomes surprises. We know that (1) was unpopular, that's why we changed it. We now know that (3) is unpopular at least in some circles (Mark Harrison can't be the only one who doesn't like it). Changing to (2) wouldn't do much to address this, because the default context has way more precision than float, so it still shows a lot of extraneous digits.

Yes, it's trivial to get rid of those extra digits, just use quotes. But if
*my* proposal were adopted, it would be trivial for numerical experts to get
the extra digits -- just use from_float(). At this point, my claim is that
we're talking essentially about what is the better experience for most
users, and while I am not much of a user of Decimal myself, I believe that
my proposal has benefits more people and situations than it has downsides.

If you write Decimal(1.1) and are surprised by the result then you have misunderstood something. It may be that you have little understanding of the difference between binary and decimal floating point (but then why are you using Decimal?). Perhaps you don't fully understand the literal->float->Decimal pathway that occurs when the expression is evaluated because you're new to Python or just haven't really thought about it before.

In any case if the result of Decimal(1.1) surprises you then it's because you're expecting it do something that should be done in a different way. Hiding the extra digits does not help a user to understand how to use Decimal.

I actually use this in teaching to demonstrate how binary floating point works. I think it's important when teaching my students for them to understand that the following is a lie:

```
>>> a = 1.1
>>> a
1.1
```

The helpful digit-hiding repr is lying to you. There is no float with the value 1.1. I can sort-of demonstrate this with some arithmetic:

```
>>> 0.1 + 0.11 - 0.11 - 0.1
1.3877787807814457e-17
```

But that gives the misleading impression that inexact arithmetic is the source of the error. It's important to understand that the error occurs straight away in the literal "1.1". I demonstrate this by showing that

```
>>> from decimal import Decimal
>>> a = 1.1
>>> Decimal(a)
Decimal('1.100000000000000088817841970012523233890533447265625')
```

which also shows the true value stored in the float.

The fact that I use this in teaching is not supposed to serve as an argument for keeping it (I could just as easily use from_float). My point is that showing the digits helps someone to understand what is going on. If a user is converting float to Decimal without knowing what they're doing then the extra digits are a clue that they don't fully understand what's happening and haven't necessarily used the best approach.

There is a good solution to the problem of non-experts wanting to write 1.1 and get the exact value 1.1: decimal literals. With that they can just write 1.1d and not need to learn any more about it. Earlier in this thread you reject that idea saying that you can't teach it to "newbies": ''' Maybe we can add a new literal notation meaning 'decimal'; that would be a relatively straightforward change to the parser (once the new decimal extension module is incorporated), but it would not do much to avoid surprising newbies (certainly you can't go teaching them to always write 3.14d instead of 3.14). However, it would probably help out frequent users of Decimal. (Then again, they might not be using literals that much -- I imagine the source of most such programs is user or file input.) ''' I disagree. If you're at the point of wanting to use the Decimal module then you're at the point where it's reasonable to learn about Decimal literals.

Also I've written code using the decimal module for high precision calculation and it has lots of decimal literals. That is I do (and I see other's doing)

```
>>> from decimal import Decimal as D
>>> d = D('1e-20')
```

even though this is never used in the decimal documentation. This is the closest you can get to decimal literals right now.

Oscar

6:59 p.m.

Thanks Oscar, that's a very well-reasoned post.

On Sat, Mar 8, 2014 at 10:11 AM, Oscar Benjamin

oscar.j.benjamin@gmail.comwrote:

On 8 March 2014 16:54, Guido van Rossum guido@python.org wrote:

I'll try to respond to Mark Dickinson's second message (and nothing else that happens in the thread since last night), because (a) it concisely summarizes his position and (b) brings up a new strawman.

On Sat, Mar 8, 2014 at 2:01 AM, Mark Dickinson dickinsm@gmail.com wrote:

On Sat, Mar 8, 2014 at 9:43 AM, Mark Dickinson dickinsm@gmail.com wrote:

I see three sane options for float to Decimal conversion:

- Raise an exception.
- Round to the nearest Decimal value using the current context for that round operation.
- Do what we're currently doing, and do an exact conversion.

I think you're writing this entirely from the POV of an expert in floating point. And I'm glad we have experts like you around! I don't consider myself

insight the experience of non-expert users.

Standards compliance is important though. Mark is essentially restating something that is repeated hundreds of times throughout various floating point standards documents: a result may be exact or it must be correctly rounded according to the current context (with appropriate flags set and traps fired).

I have mixed feelings about such standards. I can see its importance. But like the Unicode standard, it seems to want to grab authority over areas that I think belong to the language design. Also at this point claiming "compliance" with some standard is usually a morass of weasel-words rather than clearly implementing a spec.

It is not mandated that results requiring more precision than the current context be exact. It is mandated that if a result is to be inexact then the implementation must use a very specific type of inexactness. I don't believe that a standards-compliant decimal module has any wiggle room to invent a new kind of rounding (which I think would be required to achieve what you suggest).

That would be unfortunate.

When a non-expert writes Decimal(1.1), each of the three above outcomes surprises. We know that (1) was unpopular, that's why we changed it. We now know that (3) is unpopular at least in some circles (Mark Harrison can't be the only one who doesn't like it). Changing to (2) wouldn't do much to address this, because the default context has way more precision than float, so it still shows a lot of extraneous digits.

Yes, it's trivial to get rid of those extra digits, just use quotes. But
if
*my* proposal were adopted, it would be trivial for numerical experts to
get
the extra digits -- just use from_float(). At this point, my claim is
that
we're talking essentially about what is the better experience for most
users, and while I am not much of a user of Decimal myself, I believe
that
my proposal has benefits more people and situations than it has
downsides.

If you write Decimal(1.1) and are surprised by the result then you have misunderstood something. It may be that you have little understanding of the difference between binary and decimal floating point (but then why are you using Decimal?). Perhaps you don't fully understand the literal->float->Decimal pathway that occurs when the expression is evaluated because you're new to Python or just haven't really thought about it before.

Ah, but I'm not surprised. I'm unsatisfied. I understand what led to the result, but it's still not what I want, and it's a pain to train myself to do the extra thing that gives me what I want.

In any case if the result of Decimal(1.1) surprises you then it's because you're expecting it do something that should be done in a different way. Hiding the extra digits does not help a user to understand how to use Decimal.

But does showing the extra digits do anything to help? It's just as likely to teach them a trick (add quotes or a str() call) without any new understanding.

I actually use this in teaching to demonstrate how binary floating point works. I think it's important when teaching my students for them to understand that the following is a lie:

```
>>> a = 1.1
>>> a
1.1
```

The helpful digit-hiding repr is lying to you. There is no float with the value 1.1. I can sort-of demonstrate this with some arithmetic:

```
>>> 0.1 + 0.11 - 0.11 - 0.1
1.3877787807814457e-17
```

But that gives the misleading impression that inexact arithmetic is the source of the error. It's important to understand that the error occurs straight away in the literal "1.1". I demonstrate this by showing that

```
>>> from decimal import Decimal
>>> a = 1.1
>>> Decimal(a)
Decimal('1.100000000000000088817841970012523233890533447265625')
```

which also shows the true value stored in the float.

The fact that I use this in teaching is not supposed to serve as an argument for keeping it (I could just as easily use from_float). My point is that showing the digits helps someone to understand what is going on. If a user is converting float to Decimal without knowing what they're doing then the extra digits are a clue that they don't fully understand what's happening and haven't necessarily used the best approach.

I don't think every user of Decimal necessarily needs to be an expert capable of explaining what's going on. Certainly that's not needed to be an effective user of float -- the anomalies are explained to most people's satisfaction by some hand-waving about imprecise results.

There is a good solution to the problem of non-experts wanting to write 1.1 and get the exact value 1.1: decimal literals. With that they can just write 1.1d and not need to learn any more about it. Earlier in this thread you reject that idea saying that you can't teach it to "newbies": ''' Maybe we can add a new literal notation meaning 'decimal'; that would be a relatively straightforward change to the parser (once the new decimal extension module is incorporated), but it would not do much to avoid surprising newbies (certainly you can't go teaching them to always write 3.14d instead of 3.14). However, it would probably help out frequent users of Decimal. (Then again, they might not be using literals that much -- I imagine the source of most such programs is user or file input.) ''' I disagree. If you're at the point of wanting to use the Decimal module then you're at the point where it's reasonable to learn about Decimal literals.

You're right that I dismissed it too quickly. 3.14d is clearly even better than Decimal(3.14) doing the right thing. It is also still a lot more work (touches many parts of the code rather than just the Decimal class).

Also I didn't realize that the C-implemented decimal module was already used in CPython (so I thought it would be even more work).

Also I've written code using the decimal module for high precision calculation and it has lots of decimal literals. That is I do (and I see other's doing)

```
>>> from decimal import Decimal as D
>>> d = D('1e-20')
```

even though this is never used in the decimal documentation. This is the closest you can get to decimal literals right now.

Right.

But I still have this nagging feeling that the precision Decimal(<float>)
currently gives you is, in a sense, *fake*, given that the input has much
less precision.

-- --Guido van Rossum (python.org/~guido)

8:32 p.m.

On 8 March 2014 18:59, Guido van Rossum guido@python.org wrote:

On Sat, Mar 8, 2014 at 10:11 AM, Oscar Benjamin oscar.j.benjamin@gmail.com wrote: >

On 8 March 2014 16:54, Guido van Rossum guido@python.org wrote:

If you write Decimal(1.1) and are surprised by the result then you have misunderstood something.

Ah, but I'm not surprised. I'm unsatisfied. I understand what led to the result, but it's still not what I want, and it's a pain to train myself to do the extra thing that gives me what I want.

That "you" wasn't directed at *you* personally (and neither are the ones
below).

If you're using the Decimal module then it is precisely because you do need to care about rounding/accuracy etc. If you want to use it but aren't prepared to take the effort to be careful about how to pass exact input to the constructor then you're basically taking an impossible position that no one else can help you with.

In any case if the result of Decimal(1.1) surprises you then it's because you're expecting it do something that should be done in a different way. Hiding the extra digits does not help a user to understand how to use Decimal.

But does showing the extra digits do anything to help? It's just as likely to teach them a trick (add quotes or a str() call) without any new understanding.

It's enough to make you think "Why did that happen?". It's clear when you see those digits that you have not created an object with the exact value that you wanted. The obvious next question is "How do I make it do what I want?". The docs can lead you very quickly to the correct way of doing it: http://docs.python.org/3.4/library/decimal.html#quick-start-tutorial

There is a good solution to the problem of non-experts wanting to write 1.1 and get the exact value 1.1: decimal literals. With that they can just write 1.1d and not need to learn any more about it.

You're right that I dismissed it too quickly. 3.14d is clearly even better than Decimal(3.14) doing the right thing. It is also still a lot more work (touches many parts of the code rather than just the Decimal class).

Also I didn't realize that the C-implemented decimal module was already used in CPython (so I thought it would be even more work).

As Stefan mentioned earlier there are other issues to resolve around how a decimal literal should work. It's not obvious that there should be a straight-forward equivalence between 3.14d and D('3.14'). Perhaps there should be new thread to consider how to potentially do that (and whether or not it's worth it).

But I still have this nagging feeling that the
precision Decimal(<float>)
currently gives you is, in a sense, *fake*, given that the input has much
less precision.

That depends what you use it for. My most common reason for converting a float to a Decimal is to test the accuracy of a float-based calculation by comparing it against the corresponding much higher-precision decimal calculation e.g.:

```
with localcontext() as ctx:
ctx.prec = 100
error = f(D(x)) - D(f(x))
```

For this I want the constructor to give me the exact value of the float x.

Oscar

9:59 p.m.

On 03/08/2014 12:32 PM, Oscar Benjamin wrote:

On 8 March 2014 18:59, Guido van Rossum wrote: >

But I still have this nagging feeling that the
precision Decimal(<float>)
currently gives you is, in a sense, *fake*, given that the input has much
less precision.

That depends what you use it for. My most common reason for converting a float to a Decimal is to test the accuracy of a float-based calculation by comparing it against the corresponding much higher-precision decimal calculation e.g.:

```
with localcontext() as ctx:
ctx.prec = 100
error = f(D(x)) - D(f(x))
```

For this I want the constructor to give me the exact value of the float x.

I am not a mathematician, and it's been a long time since I took physics, but I seem to remember that a lot of importance was placed on significant digits.

So, how is this justified?

Python 3.4.0b3+ (default:aab7258a31d3, Feb 7 2014, 10:48:46) [GCC 4.7.3] on linux Type "help", "copyright", "credits" or "license" for more information. --> from decimal import Decimal as D --> 9017.0109812864350067128347806 9017.010981286436 --> D(9017.0109812864350067128347806) Decimal('9017.01098128643570817075669765472412109375')

In case my question isn't obvious, the direct float got me 16 digits, while the Decimal float got me 42 digits. Why is the Decimal more "accurate" that the float it came from?

-- ~Ethan~

9 Mar
9 Mar

7:32 a.m.

On Sat, Mar 08, 2014 at 01:59:32PM -0800, Ethan Furman wrote:

Python 3.4.0b3+ (default:aab7258a31d3, Feb 7 2014, 10:48:46) [GCC 4.7.3] on linux Type "help", "copyright", "credits" or "license" for more information. --> from decimal import Decimal as D --> 9017.0109812864350067128347806 9017.010981286436 --> D(9017.0109812864350067128347806) Decimal('9017.01098128643570817075669765472412109375')

In case my question isn't obvious, the direct float got me 16 digits, while the Decimal float got me 42 digits. Why is the Decimal more "accurate" that the float it came from?

That's an interesting question. It does appear to be a lot more digits than needed. Here's that float exactly, in hex:

py> 9017.010981286436.hex() '0x1.19c8167d5b50ep+13'

Let's convert it to decimal:

py> digits = list('19c8167d5b50e') py> for i, c in enumerate(digits): ... digits[i] = '0123456789abcdef'.index(c) ... py> d = Decimal(0) py> for n in digits[::-1]: ... d += n ... d /= 16 ... py> d += 1 py> d *= 2**13 py> d Decimal('9017.010981286435708170756694')

Note that this round-trips back to float correctly:

py> assert float(d) == 9017.010981286436 py>

So I must admit, I'm not sure where the extra digits come from:

Decimal('9017.01098128643570817075669765472412109375') Decimal('9017.010981286435708170756694')

but perhaps my calculation was rather naive.

-- Steven

12:36 p.m.

Steven D'Aprano steve@pearwood.info wrote:

Note that this round-trips back to float correctly:

py> assert float(d) == 9017.010981286436

Many inexact values map to the same float.

So I must admit, I'm not sure where the extra digits come from:

Decimal('9017.01098128643570817075669765472412109375') Decimal('9017.010981286435708170756694')

Without looking at your calculation, I guess it is rounding:

9017.010981286436.as_integer_ratio() (2478577105427079, 274877906944) Decimal(2478577105427079) / 274877906944 Decimal('9017.010981286435708170756698') getcontext().prec = 100 Decimal(2478577105427079) / 274877906944 Decimal('9017.01098128643570817075669765472412109375')

Stefan Krah

2:28 p.m.

On 03/09/2014 07:36 AM, Stefan Krah wrote:

Steven D'Apranosteve@pearwood.info wrote:

Note that this round-trips back to float correctly:

py> assert float(d) == 9017.010981286436

Many inexact values map to the same float.

To get the same result as Stevens calculation, the context needs to be set to 16.

from decimal import * setcontext(Context(16)) Decimal(2478577105427079) / 274877906944 Decimal('9017.010981286436')

When converting a float to Decimal... shouldn't the smallest (closest to zero) value that round trips be correct? It seems like anything else is an implicit rounding up. We don't add .5 when we convert an int to a float.

The rounding may be due to the financial need to average out small errors rather than have them accumulate. ie.. ROUND_HALF_EVEN,

from decimal import * getcontext() Context(prec=28, rounding=ROUND_HALF_EVEN, Emin=-999999, Emax=999999, capitals=1, clamp=0, flags=[], traps=[InvalidOperation, DivisionByZero, Overflow])

Which is why it's suggested decimal should only be used for financial calculations. But it's only the context that differs I think. Decimals default context does seem to be for financial calculations. Maybe there should be some easy to set contexts could be added.

```
setcontext(MATH)
setcontext(FINANCE)
setcontext(SCIENCE)
```

Or even...

```
setcontext(CALCULATOR)
```

So I must admit, I'm not sure where the extra digits come from:

Decimal('9017.01098128643570817075669765472412109375') Decimal('9017.010981286435708170756694')

Without looking at your calculation, I guess it is rounding:

> 9017.010981286436.as_integer_ratio() (2478577105427079, 274877906944) > Decimal(2478577105427079) / 274877906944 Decimal('9017.010981286435708170756698') > getcontext().prec = 100 > Decimal(2478577105427079) / 274877906944 Decimal('9017.01098128643570817075669765472412109375')

Something that bothers me about the decimal module is how the rest of the numbers are converted. (Not the ones entered in by hand.)

If we should enter x = Decimal('some number') to get the exact value, how does Decimal get y in xy = x * y if y is a float? Would it be the same as Decimal(y)? (Without quotes)

Cheers, Ron

3:52 p.m.

[Ethan Furman]

Python 3.4.0b3+ (default:aab7258a31d3, Feb 7 2014, 10:48:46) [GCC 4.7.3] on linux Type "help", "copyright", "credits" or "license" for more information. --> from decimal import Decimal as D --> 9017.0109812864350067128347806 9017.010981286436 --> D(9017.0109812864350067128347806) Decimal('9017.01098128643570817075669765472412109375')

In case my question isn't obvious, the direct float got me 16 digits, while the Decimal float got me 42 digits. Why is the Decimal more "accurate" that the float it came from?

[Steven D'Aprano]

That's an interesting question. It does appear to be a lot more digits than needed. Here's that float exactly, in hex:

py> 9017.010981286436.hex() '0x1.19c8167d5b50ep+13'

Let's convert it to decimal:

py> digits = list('19c8167d5b50e') py> for i, c in enumerate(digits): ... digits[i] = '0123456789abcdef'.index(c) ... py> d = Decimal(0) py> for n in digits[::-1]: ... d += n ... d /= 16 ... py> d += 1 py> d *= 2**13 py> d Decimal('9017.010981286435708170756694')

,,, So I must admit, I'm not sure where the extra digits come from:

Decimal('9017.01098128643570817075669765472412109375') Decimal('9017.010981286435708170756694')

but perhaps my calculation was rather naive.

The calculation is fine, but recall that decimal calculations are limited to 28 significant digits by default. It's no coincidence that len('9017.010981286435708170756694') == 29 (28 digits + a decimal point). The answer has "suffered" multiple rounding errors to make it fit in 28 digits.

Boost the context precision before you start, and you'll get more digits. Keep boosting it, and you'll eventually get 9017.01098128643570817075669765472412109375 followed by (zero or more) trailing zeroes.

A more direct way is to view this as an integer problem: mathematically,

0x1.19c8167d5b50ep+13 is
0x119c8167d5b50e / 2**(4*13 - 13) =
0x119c8167d5b50e / 2**39 = (multiplying top and bottom by 5**39)
0x119c8167d5b50e * 5**39 / 10**39

and now it's in the form of an integer numerator divided by a power of

- The numerator is:

0x119c8167d5b50e * 5**39 9017010981286435708170756697654724121093750

2:59 p.m.

On Sat, 08 Mar 2014 13:59:32 -0800 Ethan Furman ethan@stoneleaf.us wrote:

So, how is this justified?

Both representations need to uphold the round-tripping property:

float(str(some_float)) == some_float Decimal(str(some_decimal)) == some_decimal

However, since Decimal has arbitrary precision (as opposed to a fixed number of bits), the number of digits needed in the repr() can be much higher in order to uphold the round-tripping property:

float('1.1') == 1.1 True decimal.Decimal('1.1') == decimal.Decimal(1.1) False

Moreover, the Decimal constructor strives to uphold the property that Decimal(some_number) == some_number (using the required internal precision).

Therefore, decimal.Decimal(1.1) is the exact decimal value of the
binary floating-point number *best approaching* the decimal literal '1.1':

decimal.Decimal(1.1) Decimal('1.100000000000000088817841970012523233890533447265625') decimal.Decimal('1.100000000000000088817841970012523233890533447265625') == 1.1 True

Indeed, the difference is less than a binary floating-point mantissa's resolution:

decimal.Decimal(1.1) - decimal.Decimal('1.1') Decimal('8.881784197001252323389053345E-17') 2**(-sys.float_info.mant_dig) 1.1102230246251565e-16

However, since Decimal has such a high potential resolution, changing even one faraway digit will break the equality:

decimal.Decimal('1.100000000000000088817841970012523233890533447265626') == 1.1 False

(I changed the trailing '5' to a '6')

Which is why Decimal(1.1) has to be so precise, and so has its repr() too.

Of course, if you convert those literals to floats, they turn out to convert to the same binary floating-point number:

1.1 == 1.100000000000000088817841970012523233890533447265625 True

float() can therefore afford to have a "user-friendly" repr() in some cases where Decimal can't.

Regards

Antoine.

3:19 p.m.

On 03/09/2014 09:59 AM, Antoine Pitrou wrote:

Therefore, decimal.Decimal(1.1) is the exact decimal
value of the
binary floating-point number*best approaching* the decimal literal '1.1':

decimal.Decimal(1.1) Decimal('1.100000000000000088817841970012523233890533447265625') decimal.Decimal('1.100000000000000088817841970012523233890533447265625') == 1.1 True

Indeed, the difference is less than a binary floating-point mantissa's resolution:

Yes, I realized that using the number closest to zero, is an implicit rounding down, after I posted.

The increases resolution accounted for the slightly less results in the examples. But because it was using the value closest to the decimal representation (the quoted value) instead of the exact float value. The higher resolution might be slightly higher or lower in that case and have extra, or even possibly (but more rarely) fewer digits. (is that right?)

The decimal is more accurate in that manner. But more picky about quality too. Looks like there is a way to set that in the context.. So that (value +- some error amount) is used in the equality tests. Anyway need to go... Check back later this evening.

Cheers, Ron

2:19 a.m.

[Oscar Benjamin oscar.j.benjamin@gmail.com]

... I don't believe that a standards-compliant decimal module has any wiggle room to invent a new kind of rounding (which I think would be required to achieve what you suggest).

That would be true if the standard explicitly forbade extensions (any
optional operation or behavior not specified by the standard). But
there are no standards like that ;-) To comply with a standard, there
just needs to be _some_ way to "spell" all the behaviors the standard
mandates. For example, in a programming language no relevant 754-ish
standard says that the infix binary operator "*" has to implement
754-ish multiplication. So far as standard compliance goes, "*" could
mean "ignore the second argument and add 1.7 to the first argument,
rounding toward 7" - just so long as there's *some* way to spell
"754-ish multiplication".

Python (or any other implementation) is quite free to invent all the rounding modes (or any other kinds of operations, or modifications to standard behaviors) it likes. That kind of experimentation is where ideas for the _next_ iteration of the standard comes from.

2:26 a.m.

On Saturday, March 8, 2014 8:19:54 PM UTC-6, Tim Peters wrote

That would be true if the standard explicitly forbade extensions (any

optional operation or behavior not specified by the standard). But there are no standards like that ;-) To comply with a standard, there just needs to be _some_ way to "spell" all the behaviors the standard mandates.

The IEEE 854 1987 is just such a standard (Mike Cowlishaw's extension).

The generalized IEEE 754 2008 standard might not have as much wiggly room.

2:48 a.m.

[Tim]

That would be true if the standard explicitly forbade extensions (any optional operation or behavior not specified by the standard). But there are no standards like that ;-) To comply with a standard, there just needs to be _some_ way to "spell" all the behaviors the standard mandates.

[Mark H. Harris]

The IEEE 854 1987 is just such a standard (Mike Cowlishaw's extension).

The generalized IEEE 754 2008 standard might not have as much wiggly room.

I have the 2008 version of the standard; it's thoroughly ordinary in these respects.

As usual, it's not even "a language" that's normally said to be compliant with the standard. Instead:

```
Conformance to this standard is a property of a specific
implementation of a specific programming environment, rather
than of a language specification.
```

Although:

```
However a language standard could also be said to conform to this standard
if it were constructed so that every conforming implementation of
```

that language also conformed automatically to this standard.

So far as languages go, there's very little a standard of this nature _can_ say! Languages vary enormously in syntax and conventions. Indeed, for example, the standard "spells" multiplication as

```
multiplication(x, y)
```

That doesn't mean a conforming implementation has to supply a 2-argument function named "multiplication". It just means a conforming implementation has to supply _some_ way to spell what the standard defines via "multiplication(x, y)".

Etc.

8 Mar
8 Mar

6:49 p.m.

On Sat, Mar 8, 2014 at 4:54 PM, Guido van Rossum guido@python.org wrote:

That at least could be addressed by making the default context one that
corresponds to IEEE's decimal64, which has a precision of 16 decimal
digits; there doesn't seem to be any particularly good rationale for the
current choice of default context. And that would address your point
(which I agree with) that `Decimal(x)`

currently shows way more digits than
are useful. It wouldn't address Mark Harris's needs, though, and would in
some sense make things worse for non-expert users: for *most* floats x,
Decimal(x) would then give what the user expects purely because 16 digits
is a good match for the binary float precision, but for a small fraction of
inputs it would give a surprising result.

from decimal import Decimal, getcontext

getcontext().prec = 16

+Decimal(1.1) # + to simulate rounding to 16 digits

Decimal('1.100000000000000')

+Decimal(1.2)

Decimal('1.200000000000000')

+Decimal(9.7)

Decimal('9.699999999999999')

This feels like a strawman, since Decimal(2**1000 + 1) and

Decimal(2.0**1000 + 1) produce different outcomes (the latter gives a lot of digits but is one too small), and the similar examples with a large exponent (10000) differ dramatically (the float version raises an exception).

For most purposes it's a fluke that 2.0**1000 can be represented exactly as a float, and the argument doesn't convince me at all. There are just too many odd examples like that, and it always will remain a minefield.

Accepted, but I believe the proposal would break a number of other
expectations too with respect to ints and floats. For one, we currently
have the unsurprising, and I believe desirable, property that conversion to
Decimal is monotonic: for any finite numbers (int, float or Decimal) x and
y, if x <= y then Decimal(x) <= Decimal(y). The proposal would break that
property, too: you could find examples of an integer x and float y such
that `x < y`

and `Decimal(x) > Decimal(y)`

would be
simultaneously True.

x = 49534541648432951

y = x + 2.0

x < y

True

Decimal(x) > Decimal(repr(y)) # simulating proposed meaning of Decimal(y)

True

I'm still struggling a bit to express exactly what it is that bothers me so much about the proposal. It's a combination of the above with:

there's an obvious

*correct*way to convert any value to Decimal, and that's to round to the context precision using the context rounding mode - that's what's done almost universally for Decimal arithmetic operations; it feels wrong that something as direct as`Decimal(x)`

would do anything else (the current exact conversions actually*do*rankle with me a bit, but this is what I'd replace them with)this proposal would not play well with other binary number formats if they were ever introduced: if we introduced a float128 type (or a float32 type), it would be awkward to reconcile the proposed conversion with a conversion from float128 to Decimal. That's mostly because the semantics of the proposed conversion use information about the

*format*of the input type, which is unusual for floating-point and floating-point standards: operations tend to be based purely on the*values*of the inputs, disregarding the formats.if we're aiming to eliminate surprises, the 'fix' doesn't go far enough: Decimal(1.1 + 2.2) will still surprise, as will Decimal(0.12345123451234512345)

contrarily, the fix goes too far: it feels wrong to be changing the semantics of float to Decimal conversion when the real problem is that the user wants or expects floating-point literals to represent decimal numbers.

If the proposal goes forward, I'll live with it, and will simply avoid
using the `Decimal`

type to convert from floats or ints. But I'd really
prefer to keep the short Decimal(x) spelling as something simple and
non-magic, and find more complicated ways (quotes!) of spelling the magic
stuff.

I wonder what Cowlishaw would say about our current discussion. He is also

the father of Rexx...

I wonder that, too.

I've said too much already; beyond registering my strong -1 on this proposal, I'm going to keep out of further discussion.

Mark

6:55 p.m.

On Sat, 8 Mar 2014 18:49:02 +0000 Mark Dickinson dickinsm@gmail.com wrote:

If the proposal goes forward, I'll live with it, and
will simply avoid
using the `Decimal`

type to convert from floats or ints. But I'd really
prefer to keep the short Decimal(x) spelling as something simple and
non-magic, and find more complicated ways (quotes!) of spelling the magic
stuff.

For the record, as a non-float expert, Mark's arguments convince me.

Just my 2 cents ;-)

Regards

Antoine.

9:32 p.m.

On 8 March 2014 18:55, Antoine Pitrou solipsis@pitrou.net wrote:

On Sat, 8 Mar 2014 18:49:02 +0000 Mark Dickinson dickinsm@gmail.com wrote: >

If the proposal goes forward, I'll live with it,
and will simply avoid
using the `Decimal`

type to convert from floats or ints. But I'd really
prefer to keep the short Decimal(x) spelling as something simple and
non-magic, and find more complicated ways (quotes!) of spelling the magic
stuff.

For the record, as a non-float expert, Mark's arguments convince me.

Just my 2 cents ;-)

I am in the same position. I "instinctively" wanted Decimal(1.1) to return the same value as Decimal('1.1'), but as I read Mark's argument, it became clearer to me that doing so involves a deliberate loss of precision that is not present anywhere else in the chain of conversions.

literal 1.1 -> float 1.1 is a precise conversion, in the sense that no other float better represents the (decimal) literal 1.1. The current float 1.1 -> decimal conversion is exact.

Converting float 1.1 to Decimal('1.1') deliberately drops precision, even though an exact conversion is possible. I don't think that this is something that should happen implicitly.

On the other hand, the *reason* people do Decimal(1.1) is because they
want a decimal value of 1.1. (Doh.) The fact that this isn't what they
are actually calculating is a mistake they make because Python doesn't
make it easy to *get* the decimal value 1.1 (I know the correct
approach is Decimal('1.1'), but I don't think that is obvious or
intuitive[1]). Decimal literals (1.1d) would resolve this issue
without introducing implicit rounding operations.

Paul

[1] The reason Decimal('1.1') feels non-obvious to me is that it feels
like it's getting to a decimal "via" a string. Of course, Decimal(1.1)
is also a going "via" a float. The *value* of the current behaviour is
that it leads people to an understanding that this is what it is.

7:13 p.m.

On Saturday, March 8, 2014 12:49:02 PM UTC-6, Mark Dickinson wrote:

- if we're aiming to eliminate surprises, the 'fix' doesn't go far enough:
Decimal(1.1 + 2.2) will still surprise, as will {snip}

Correct. That is why a decimal literal notation is also needed. I've emulated it here:

from pdeclib import *
sqrt(1.1+2.2)**2
Decimal('3.30000000000000026645352591003756970167159') <==== this is a
surprise, to a naive user
sqrt(d(1.1)+d(2.2))**2
Decimal('3.29999999999999999999999999999999999999999') <=== this if
decimal literal,

```
sqrt(1.1d + 2.2d)
```

```
But, you are correct that what is "really" wanted
--someday-- is to
```

have the literal be decimal (rather than float) to begin with.

Neither here nor there, step at a time over time is better than simple status quo.

Please let me be clear, I think Guido's proposal is a very good first step. It makes sense for the most users (esp naive ones) and does not interfere with advanced users.

marcus

8:12 p.m.

On Sun, Mar 9, 2014 at 6:13 AM, Mark H. Harris harrismh777@gmail.com wrote:

```
But, you are correct that what is "really"
wanted --someday-- is to
```

have the literal be decimal (rather than float) to begin with.

Neither here nor there, step at a time over time is better than simple status quo.

Please let me be clear, I think Guido's proposal is a very good first step. It makes sense for the most users (esp naive ones) and does not interfere with advanced users.

It's probably time someone [1] wrote up a PEP about all this. The most important points, as I see it, are:

1) Create a new Decimal literal notation - 1.234d seems to be the most popular syntax. This is reasonably uncontroversial, but it has consequences. 2) Create a new float literal notation - 1.234f or 1.234r or any of the other proposals. 3) Possibly change repr(float) to include the tag. 4) Introduce a "from __future__ import decimal_literals" (named to parallel unicode_literals - you can get a u"literal" without that directive, but the default literal type becomes unicode) 5) What about int/int? Should that now be Decimal? Should it be per-module??? 6) Further cans of worms like #5

Introducing 1/2/4 would let you stick the future directive into PYTHONSTARTUP and then run Python as an interactive decimal calculator.

ChrisA

[1] Not it!

10:15 p.m.

On Mar 8, 2014, at 12:12, Chris Angelico rosuav@gmail.com wrote:

It's probably time someone [1] wrote up a PEP about all this.

The most important points, as I see it, are:

1) Create a new Decimal literal notation - 1.234d seems to be the most popular syntax. This is reasonably uncontroversial, but it has consequences.

1a) Move the Decimal type's implementation out of the module and into Objects, or make it a frozen module that's always loaded at startup, or invent some scheme to load it as needed.

1b) Possibly move the type to builtins? (Certainly not necessary--function is a built-in type, and you can create them without importing anything, but the type lives in types, not builtins.)

1c) Possibly add the implementation to the public C API? (With which methods? FromString, but what about From/AsDouble? Is there any need to From/AsTuple from C? Inspect the object in any other way? Get/set the current context?)

2) Create a new float literal notation - 1.234f or 1.234r or any of the other proposals. 3) Possibly change repr(float) to include the tag.

3a) Change repr(Decimal) to use the new literal.

4) Introduce a "from __future__ import decimal_literals" (named to parallel unicode_literals - you can get a u"literal" without that directive, but the default literal type becomes unicode) 5) What about int/int? Should that now be Decimal? Should it be per-module??? 6) Further cans of worms like #5

Introducing 1/2/4 would let you stick the future directive into PYTHONSTARTUP and then run Python as an interactive decimal calculator.

10:25 p.m.

On Sun, Mar 9, 2014 at 9:15 AM, Andrew Barnert abarnert@yahoo.com wrote:

3) Possibly change repr(float) to include the tag.

3a) Change repr(Decimal) to use the new literal.

I'd put that in the consequences of part 1 - it's pretty obvious that, with a decimal literal, the repr of a Decimal should be that. It'd affect people's tests but shouldn't break much else. But changing repr(float) means tagging everything, rather than just having a new convenient notation for Decimal. It's the same physical/technical change but it has a lot more implication :)

And yeah. You listed several significant quinseconses, and I've no doubt there'll be others.

ChrisA

10:36 p.m.

On 8 March 2014 20:12, Chris Angelico rosuav@gmail.com wrote: >

It's probably time someone [1] wrote up a PEP about all this. The most important points, as I see it, are:

1) Create a new Decimal literal notation - 1.234d seems to be the most popular syntax. This is reasonably uncontroversial, but it has consequences.

I feel like I've boxed myself into the corner by arguing that
*someone* should do this. :)

Unless someone else especially wants the mantle I'll try to write a PEP about decimal literals (I won't have any real time until Monday though).

Oscar

8:16 p.m.

On Saturday, March 8, 2014 12:49:02 PM UTC-6, Mark Dickinson wrote:

- if we're aiming to eliminate surprises, the 'fix' doesn't go far enough: Decimal(1.1 + 2.2) will still surprise, as will {snip}

On the other hand, how is this now possible?

====== RESTART ========================== from pdeclib import * d(1.1+2.2) Decimal('3.3') sqrt(1.1+2.2) Decimal('1.81659021245849499925351968583091621951684') sqrt(1.1+2.2)**2 Decimal('3.29999999999999999999999999999999999999999')

def sqrt(x): """ sqrt(x) square root function

```
(x may be string, int, float, or decimal)
(returns decimal rounded to context precision)
"""
y=x.__round__(15)
with localcontext(ctx=None) as cmngr:
cmngr.prec+=14
sqr=Decimal(repr(y)).sqrt()
return +sqr
```

What say you?

8:25 p.m.

On Saturday, March 8, 2014 2:23:23 PM UTC-6, Mark H. Harris wrote: > >

```
y=x.__round__(15) <====== how to
set the round within context so
```

that works for float literals & decimals with localcontext(ctx=None) as cmngr: {snip}

Is this possible..?

Only do this for float literals. Yes? Set a trap, and choose.

I gotta play with this a bit...

9:01 p.m.

On Saturday, March 8, 2014 2:16:00 PM UTC-6, Mark H. Harris wrote: > > >

On Saturday, March 8, 2014 12:49:02 PM UTC-6, Mark Dickinson wrote:

- if we're aiming to eliminate surprises, the 'fix' doesn't go far enough: Decimal(1.1 + 2.2) will still surprise, as will {snip}

How about this then: I think I've got it...

=====
RESTART =========================
from pdeclib import *
d(1.1+2.2)
Decimal('3.3')
sqrt(1.1+2.2)**2
Decimal('3.29999999999999999999999999999999999999999')
x=d(1.1)
y=d(2.2)
sqrt(x+y)**2
Decimal('3.29999999999999999999999999999999999999999')

Code Below--------------------------

#
*****/

# sqrt(x) return decimal sqrt(x) rounded to context precision

#
*****/

def sqrt(x): """ sqrt(x) square root function

```
(x may be string, int, float, or decimal)
(returns decimal rounded to context precision)
"""
with localcontext(ctx=None) as cmngr:
cmngr.prec+=14
if (isinstance(x, float)==True):
y=x.__round__(15)
sqr=Decimal(repr(y)).sqrt()
else:
sqr=Decimal(x).sqrt()
return +sqr
```

-------------------- what say you? --------------------- marcus

9:35 p.m.

On Mar 8, 2014, at 13:01, "Mark H. Harris" harrismh777@gmail.com wrote:

On Saturday, March 8, 2014 2:16:00 PM UTC-6, Mark H. Harris wrote:

On Saturday, March 8, 2014 12:49:02 PM UTC-6, Mark Dickinson wrote:

- if we're aiming to eliminate surprises, the 'fix' doesn't go far enough: Decimal(1.1 + 2.2) will still surprise, as will {snip} How about this then: I think I've got it...

===== RESTART =========================
from pdeclib import *
d(1.1+2.2)
Decimal('3.3')
sqrt(1.1+2.2)**2
Decimal('3.29999999999999999999999999999999999999999')
x=d(1.1)
y=d(2.2)
sqrt(x+y)**2
Decimal('3.29999999999999999999999999999999999999999')

Code Below--------------------------

#
*****/

# sqrt(x) return decimal sqrt(x) rounded to context precision

#
*****/

def sqrt(x): """ sqrt(x) square root function

```
(x may be string, int, float, or decimal)
(returns decimal rounded to context precision)
"""
with localcontext(ctx=None) as cmngr:
cmngr.prec+=14
if (isinstance(x, float)==True):
y=x.__round__(15)
sqr=Decimal(repr(y)).sqrt()
else:
sqr=Decimal(x).sqrt()
return +sqr
```

-------------------- what say you? ---------------------

It looks like you're trying to emulate a pocket calculator here. The question is, why are you accepting floats in the first place if that's your goal?

9:54 p.m.

On Saturday, March 8, 2014 3:35:14 PM UTC-6, Andrew Barnert wrote:

It looks like you're trying to emulate a pocket calculator here. The

question is, why are you accepting floats in the first place if that's your goal?

That's fair... if it were me, or you, I wouldn't. In fact, if it were just me, or just you, I don't think any of us would be talking about this--- I wouldn't even have brought it up.

I am advocating for people--- I am going to say, " if you want just a tad more than double just to check things out, please use the decimal module and import pdeclib. These folks are what Guido called newbies, I'll call them naive users, and these folks are using a calculator (like the TI89) and they will be prone to keying in sqrt(2.345) /

Its just easier than explaining to them (at this point) all the stuff they need to understand to use the functions by making sure they feed decimals to the function; either by quoting their numeric inputs, or even using a decimal literal, &c. They know what the functions do, and will want to use them to get more digits, but may not remember the rules to get the "correct" digits. So I'm advocating for them, to make it easier.

If it were just me, I'd just make sure the functions got decimals and be done with it.

If you look at my square root function, its really not right; but its closer. For instance if you give it 1/3 it still is only slightly better than before. I don't have all the junk after the 16 th digit, but, I only have .33333333 to 15 places. So, believe me, I know there is no one truly good answer here. Just trying to make it easier for the most people.

marcus

10:27 p.m.

On Mar 8, 2014, at 13:54, "Mark H. Harris" harrismh777@gmail.com wrote:

On Saturday, March 8, 2014 3:35:14 PM UTC-6, Andrew Barnert wrote:

It looks like you're trying to emulate a pocket calculator here. The question is, why are you accepting floats in the first place if that's your goal?

That's fair... if it were me, or you, I wouldn't. In fact, if it were just me, or just you, I don't think any of us would be talking about this--- I wouldn't even have brought it up.

I am advocating for people--- I am going to say, " if you want just a tad more than double just to check things out, please use the decimal module and import pdeclib. These folks are what Guido called newbies, I'll call them naive users, and these folks are using a calculator (like the TI89) and they will be prone to keying in sqrt(2.345) /

So your goal is that if someone is a TI89 expert but a Python novice, they can easily port a function from their TI89 to Python (or maybe even write a TI89-like desktop calculator program) with your library? That doesn't seem too unreasonable of a goal.

The problem is that you're not quite achieving it. If you want a 16-digit-display calculator, using a mix of float64 values and decimals that expand as needed for intermediates is not equivalent to using 20-digit fixed decimals. So they will not get the same results for many functions that they port. And if they really care about having more than a few digits of precision, they either care about these differences, or really really should care. Hiding them in many but not all cases doesn't seem to be doing them a service.

A simpler solution might be to just only support a fixed precision of, say, 8 digits. Then you can handle float inputs without running into that one-too-few 3's problem you mentioned. But obviously this isn't great either--you'd just end up with users who say "your library is perfect for me, except that I need 10 digits, not 8"...

11:33 p.m.

On Saturday, March 8, 2014 4:27:25 PM UTC-6, Andrew Barnert wrote:

So your goal is that if someone is a TI89 expert but a Python novice, they can easily port a function from their TI89 to Python (or maybe even write a TI89-like desktop calculator program) with your library? That doesn't seem too unreasonable of a goal.

Your points are all mostly valid, taking all assumptions into account. Some of these folks actually have TI89's but don't really know how to use them yet. Some of them have TI84+, or TI83, some TI Nspire... some others. Mostly the comparison between the calc and python isn't the point. The point is using python to do the trig (or whatever) and along the way introducing programming. Its just one use case that is not too different than a thousand other "average" or naive user cases, where the folks are still on the learning curve but need to be introduced to computers and computer science as well as the maths. This will also be good for folks who know what they are doing and will now be able to do it just a little more efficiently. For folks who really know what they are doing spelling issues are not even a problem. So, we can help newbie/naive users without impacting advanced users like yourself.

The bottom line is trying to eliminate as few surprises as possible, making the learning curve as easy and fun as possible. The point is not to make a calculator; rather to make a decimal floating point package for python that is flexible. Experienced users can use it for all kinds of purposes, and newbies can use it too. The experts on this list, like you, will have no problem using the package (if you want) and folks that are brand new to python will be able to use it too.

At some point the decimal concepts will come up, and the decimal literal will come in very handy then. For folks using 3.3.4 or below obviously the caveats are going to need to be explained in the documentation... and my functions are going to need to take in floats and do something meaningful with them, as you saw in the square root routine. Its a little more hassle for me to code and maintain, but its less "surprise" impact on users will be worth it (esp for young wanna bee naiveté types).

marcus

9 Mar
9 Mar

12:21 a.m.

On Saturday, March 8, 2014 4:27:25 PM UTC-6, Andrew Barnert wrote: >

{snip}

Many folks "play" on the interactive python terminal; and some python experienced users and developers forget this. For instance the QPython people did not provide a terminal or interactive python with their first release because they were focused on their product being a script reader and forgot that normal people experiment on the interactive console, for calculation, and other things. We had a discussion the other night about the decimal distribution (numerical digit distribution) of the digits 0-9 in the number PI. The conjecture is that as the number of digits increased the distribution evens out, and PI is for all intents and purposes a random number generator (for large numbers of digits). But what does it look like to users who have never seen it, and how hard is it to code up? Well, its two lines of python code, and inexperienced users can not believe that: below:

from pdeclib import * dscale(1010) 42 pi=get_PI() sPI=repr(pi)[:1002]

for n in range(10): print(n, sPI.count(str(n)))

0 92 1 114 2 102 3 103 4 92 5 97 6 93 7 95 8 100 9 104

Well, there it is; the distribution of the digits of PI in the first 1002, precisely; all from the console.

marcus

12:35 a.m.

On Saturday, March 8, 2014 6:21:59 PM UTC-6, Mark H. Harris wrote:

Actually this is it, because I keep forgetting that repr(D) includes "Decimal " in the string, must use str()

from pdeclib import * dscale(1010) 42 pi=get_PI() sPI=str(pi)[:1002] for n in range(10): print(n, sPI.count(str(n)))

0 93 1 116 2 103 3 103 4 93 5 97 6 94 7 95 8 101 9 106

Anyway, its just fun....

5:02 a.m.

On Sat, Mar 08, 2014 at 01:54:11PM -0800, Mark H. Harris wrote:

I am advocating for people--- I am going to say, " if you want just a tad more than double just to check things out, please use the decimal module and import pdeclib. These folks are what Guido called newbies, I'll call them naive users, and these folks are using a calculator (like the TI89) and they will be prone to keying in sqrt(2.345) /

When we get newbies asking why Python does not behave exactly like Java, we explain to them that Python is not Java and describe the differences. We do not try to make Python into a clone of Java. The same applies to newies who complain that Python does not behave exactly like Ruby, or PHP, or Perl, or Lisp.

I don't think TI-89 should be the exception. I don't think it is a good idea to try to make a general purpose programming language like Python into a (mathematical) clone of the TI-89 calculator just to satisfy the tiny minority of users who are numerically naive TI-89 users.

Its just easier than explaining to them (at this point) all the stuff they need to understand to use the functions

When you put it like that, it sounds like you are arguing, not that it
is better for the users to avoid learning about floating point maths,
but that it's easier for *you* if you don't need to explain floating
point maths. That's okay. Point them to the python-list or tutor mailing
lists, and the regulars there will be more than happy to teach them, to
the best of our abilities.

by making sure they feed decimals to the function; either by quoting their numeric inputs, or even using a decimal literal, &c. They know what the functions do, and will want to use them to get more digits, but may not remember the rules to get the "correct" digits. So I'm advocating for them, to make it easier. [...] Just trying to make it easier for the most people.

Who are these people you are trying to help? The user-base for Python is developers, and people trying to learn to be developers. Python is not an end-user application. Python is a programming language, and the people using it should be treated as if they were programmers.

I think that there are very few non-programmer, numerically-naive, TI-89 users working with Python, and I think that shaping Python specifically for them is a mistake. It is completely appropriate to build an interactive calculator using Decimals aimed at the TI-89-using audience, and I applaud that. But that's an application written in Python, not Python itself.

If it were just me, I'd just make sure the functions got decimals and be done with it.

You would do that even though the majority of numeric users of Python would be opposed to such a change?

-- Steven

6:09 a.m.

On Saturday, March 8, 2014 11:02:07 PM UTC-6, Steven D'Aprano wrote:

I don't think TI-89 should be the exception. I don't think it is a good

idea to try to make a general purpose programming language like Python into a (mathematical) clone of the TI-89 calculator just to satisfy the tiny minority of users who are numerically naive TI-89 users.

hi Steven, yeah, in some ways I wish I'd never mentioned the TI89, because I've been misinterpreted (for which I am sorry). I mentioned the TI89 because it uses decimal floating points by default , I think (as I believe python should also). I mentioned the TI89 mainly as a "calculator" and that is the one that popped into my head... the reason for mentioning it is that folks who poke a number into a calculator don't quote the number, they don't add a "d" to the number, and they expect the calculator to interpret their number correctly. ( it was a usability reference )

> >

Its just easier than explaining to them (at this point) all the stuff they need to understand to use the functions

When you put it like that, it sounds like you are arguing, not that it
is better for the users to avoid learning about floating point maths,
but that it's easier for *you* if you don't need to explain floating
point maths. That's okay. Point them to the python-list or tutor mailing
lists, and the regulars there will be more than happy to teach them, to
the best of our abilities.

```
Well, its that too for sure, but its certainly more. Its a way for maths
```

and other explanations to be brought on-line in stages (learning mods) that are progressive, and yet have the students be able to get the right results from the beginning and not to be frustrated. Again, a usability thing.

Who are these people you are trying to help? The user-base for Python is developers, and people trying to learn to be developers. Python is not an end-user application. Python is a programming language, and the people using it should be treated as if they were programmers.

```
Unfortunately, I think that is the saddest thing I have heard you say
```

in almost seven years of knowing you. The earliest beginnings of Python (ABC at CWI) was to "Stamp out BASIC" . BASIC was designed at Dartmouth in 1964 to help average users and other scientists to be able to use a computer WITHOUT having to be a developer (computer scientist, nor programmer). Python has the opportunity to be the BASIC of the 21st century (and beyond). Part of the reason Python has NOT had a greater general acceptance is that it is becoming a programming language for experts. This should not generally true, fortunately. I mean that Python is actually a VERY good language for newbies to start with, and its a VERY good language for average people and other scientists to interact with a computer in sophisticated ways WITHOUT having to be a computer scientist nor programmer.

>

I think that there are very few non-programmer, numerically-naive, TI-89 users working with Python, and I think that shaping Python specifically for them is a mistake. It is completely appropriate to build an interactive calculator using Decimals aimed at the TI-89-using audience, and I applaud that. But that's an application written in Python, not Python itself.

```
Yes, and no. Again, the TI89 has really nothing to do with it... I
```

am advocating for general average usability of the language by the masses; whether they are software engineers or not, whether they are computer scientists or not, whether they are have a PhD in mathematics or not. Python should be a both | and; new folks should feel welcome and encouraged, and experienced folks should feel the power in their hands. Those ruby slippers should be the magic in the castle, and the way home for Dorothy. There are actually a large number of people trying out Python for the first time, and many professional people and academic (like Matlab or Mathematica users) who are switching to Python because of its power and flexibility (not to mention freedom). Both sets of "people" need to be welcomed and cared for. I honestly think that is one of the richest personal traits of the BDfL, and in my observation (of course he should speak for himself) he seems to advocate for a both |and (at least that is how he makes me feel about it, when I have heard him speak publicly).

If it were just me, I'd just make sure the functions got decimals and be

done with it.

You would do that even though the majority of numeric users of Python would be opposed to such a change?

```
I am not sure I understand this question. All I'm saying is that as a
```

programmer, I am
willing to take on constraints (spelling, rules, syntax, heaps of
knowledge) in stride (as a
software engineer that's my job) which I do not feel average people and
other scientists
should have to worry about. Just like at Dartmouth in 1964... a physicist
should be able
to sit down to a python terminal and in very short order (with some small
doc, tutorial) be
able to be talking to Python rapidly. The experience should not be
frustrating, the experience
should in fact be as intuitive as possible, and it might also be fun...

very seriously I believe this.

```
Python is a computer language. But, it should not be C++ (we already
```

have one of those), and it should not be Java (for heaven's sake, certainly not that) and it should not be GNU C nor Assembler, nor should it require a PhD in computer science to use it ! Python programming can and should be a rich and rewarding "liberal art" which may be learned by anyone (even children) to accomplish real world tasks with a rapid development cycle and a minimal of head banging (whatever I mean by that, its all debatable).

```
I ran into this at IBM all the time too... sooner or later developers
```

become arrogant and self-absorbed. IBM engineers are sometimes their own best customers, and their own worst enemies. I am hoping that does not happen in the Python community. Python community has a very rich and talented group of engineers, mathematicians, coders, coordinators, doc writers, pep writers, testers, and a super BDfL. It would be sad IMHO for the Python community to implode (that is, build a superstructure or play-box for developers, forgetting what they are developing, and for whom).

```
This is my philosophy, and I don't expect that everyone will share it
```

(that's just not realistic). I do hope that Python receives wider adoption in the future, and I do hope that more people (esp women, kids, young people) will choose computer science (through Python education) and that more people will actually reach out and and learn programming as a liberal art. I am sick frankly, of our point-and-grunt culture (here in the States, and elsewhere). Python is a way out of point-and-grunt, and a way forward into thinking with a computer as a liberal art--- and as fine art. It is the power of thinking in the 21st century, and it is the power of communication, collaboration, and cooperation between people and nations in the 21st century. I mean it.

marcus

> >

6:21 a.m.

I think that there are very few non-programmer, numerically-naive, TI-89 users working with Python,

students.

For numerical applications there is Numpy.

and I think that shaping Python specifically for them is a mistake. It is completely appropriate to build an interactive calculator using Decimals aimed at the TI-89-using audience, and I applaud that. But that's an application written in Python, not Python itself.

The decimal module really should only be used for financial applications; there should be a separate high-precision float type.

Another note on the 'd' notation - this might be confused with the FORTRAN 'd' for double precision number. Would you write, 1.2e3d or 1.2d3?

-Alexander

6:35 a.m.

On Sunday, March 9, 2014 12:21:07 AM UTC-6, Alexander Heger wrote:

The decimal module really should only be used for financial applications; there should be a separate high-precision float type.

No, no, no.... It may and should be used for financial calculations, so true, but it may ALSO be used for a general purpose high precision (and) arbitrary precision decimal floating point. Otherwise it might as well have been patterned after the fixed width standard. It was patterned after Cowlishaw's extension.

>

Why do you say only financial?

9:37 a.m.

Why do you say only financial?

Arbitrary precision is good - but why should it be done in a way that is not natural to how machines work? But I admit, I have not looked at the code how it is implemented. Maybe it is done well (only saving integer part and an exponent). I only recall the BCD standard used in Turbo Pascal - which was not so great - and really mostly for financial use.

But there is no point in having fractions in decimal for computation, unless the result has to have a good representation in a specific base. Something you can send to a tax office. The decimal package solves problems specific to financial. And, yes, it can be used for other applications lacking other arbitrary precision classes in Python standard class library - but it would not be my first choice for other applications.

Hence I propose to have a different arbitrary precision package for applications that do not require 'decimal' precision. As for class hierarchy it likely should be a base class for the decimal package. (Then you could even add other bases as well.)

7:10 a.m.

On Sunday, March 9, 2014 12:21:07 AM UTC-6, Alexander Heger wrote: >

{snip}

hi Alexander, Steve, this is one of Mike Cowlishaw's papers regarding decimal floating point, hardware decimal floating point, and the IEEE 854 1987 standard, and other topics. He mentions the TI89, and others, that use decimal floating point arithmetic, with citations.

http://www.cs.tufts.edu/~nr/cs257/archive/mike-cowlishaw/decimal-arith.pdf

Its a good read.

7 p.m.

On Sunday, March 9, 2014 1:10:41 AM UTC-6, Mark H. Harris wrote:

this is one of Mike Cowlishaw's papers regarding decimal floating point, hardware decimal floating point, and the IEEE 854 1987 standard, and other topics. He mentions the TI89, and others, that use decimal floating point arithmetic, with citations.

http://www.cs.tufts.edu/~nr/cs257/archive/mike-cowlishaw/decimal-arith.pdf

In the course of this thread I have mentioned Michael Cowlishaw's Rexx, and my work at IBM; for one thing adding a library of decimal floating point routines for use with VM370 Rexx which provides the scientific and transcendental functions for the now arcane 370 system. I found one of the scripts in my archive and uploaded it to code.google.com for historical purposes and perspective. If anyone is interested it can be found here:

https://code.google.com/p/rexxdecimallibrary/

Cheers

3:05 p.m.

On Sun, 9 Mar 2014 16:02:07 +1100 Steven D'Aprano steve@pearwood.info wrote:

I don't think TI-89 should be the exception. I don't think it is a good idea to try to make a general purpose programming language like Python into a (mathematical) clone of the TI-89 calculator just to satisfy the tiny minority of users who are numerically naive TI-89 users.

I'm surprised: are physical calculators (TI-89) still in use today?

Regards

Antoine.

6:53 p.m.

On Sunday, March 9, 2014 10:05:14 AM UTC-5, Antoine Pitrou wrote: >

tiny minority of users who are numerically naive TI-89 users.

I'm surprised: are physical calculators (TI-89) still in use today?

Oh my, yes. They are the premier programmable graphing calculator today (not the only one, or course) that supports advanced computation (calc, linear algebra, &c) and with not only polar and rect coord graphing but also 3D graphing. Its native language is BASIC, in the calculator supports a plethora of applications for science, business, academia, engineering, and also a host of organizer apps like calendar, notepad, spread sheet (yes, really) and more. Its a phenomenal device, really.

The TI84+ is also still marketed heavily; and it has been updated lately with a color screen. It is not sufficient for college engineering work, but its fine for high school and general programming/graphing. Its language is also BASIC.

Cheers

7:01 p.m.

On Sun, Mar 09, 2014 at 04:05:14PM +0100, Antoine Pitrou wrote:

On Sun, 9 Mar 2014 16:02:07 +1100 Steven D'Aprano steve@pearwood.info wrote:

I don't think TI-89 should be the exception. I don't think it is a good idea to try to make a general purpose programming language like Python into a (mathematical) clone of the TI-89 calculator just to satisfy the tiny minority of users who are numerically naive TI-89 users.

I'm surprised: are physical calculators (TI-89) still in use today?

I don't know about other places, but in Australia graphing calculators, including the TI-89 series, are required for all high school school students doing maths.

-- Steven

7:32 p.m.

On Sunday, March 9, 2014 2:01:46 PM UTC-5, Steven D'Aprano wrote: >

On Sun, Mar 09, 2014 at 04:05:14PM +0100, Antoine Pitrou wrote:

I'm surprised: are physical calculators (TI-89) still in use today?

I don't know about other places, but in Australia graphing calculators, including the TI-89 series, are required for all high school school students doing maths.

The same is true in the States too; well at least in Southeast Minnesota where I reside. Also, I have a son who is planning on engineering school at Iowa State next fall; they require the TI89 {preferably} or equiv for their course work.

8 Mar
8 Mar

9:33 p.m.

On Sat, Mar 8, 2014 at 10:49 AM, Mark Dickinson dickinsm@gmail.com wrote:

Accepted, but I believe the proposal would break a
number of other
expectations too with respect to ints and floats. For one, we currently
have the unsurprising, and I believe desirable, property that conversion to
Decimal is monotonic: for any finite numbers (int, float or Decimal) x and
y, if x <= y then Decimal(x) <= Decimal(y). The proposal would break that
property, too: you could find examples of an integer x and float y such
that `x < y`

and `Decimal(x) > Decimal(y)`

would be
simultaneously True.

I think that Mark Dickinson's point about breaking monotonicity is a STRONG argument against the proposal to change the meaning of Decimal(float_num) to produce Decimal(repr(float_num)). That latter spelling is already available if anyone wants it.

The focus on float value that have been entered as literals feels like a
distraction to me. Floats come from other places as well, and making
Decimal(float_num) produce the *exact* value feels far more desirable than
producing "something which rounds to the same float.

As I think Guido has acknowledged in a recent post, a far better and more
intuitive approach is just to make decimal literals easier to write.
Teaching users to write '3.14d' doesn't feel that hard to me, and it looks
notably prettier than 'decimal.Decimal("3.14")'. A *would* also like an
optional explicit binary-float literal though, e.g. '3.14f'. This wouldn't
actually mean anything different from '3.14', but then "PI" doesn't mean
anything different from u"PI" either. The optional bin-float specifier, in
principle, allows for some Python 4000 in which decimal is the default
literal (although I don't think I'd ever support that, but no harm in
letting that decision be made years from now).

- if we're aiming to eliminate surprises, the 'fix' doesn't go far enough:
Decimal(1.1 + 2.2) will still surprise, as will Decimal(0.12345123451234512345)

This is a simple illustration of why the "do what I mean" goal of the
proposed change will fall apart rather quickly for anything but the
simplest examples. The status quo of Decimal(float_num) producing the
*exact* value continues to feel the best to me.

I've said too much already; beyond registering my strong -1 on this

proposal, I'm going to keep out of further discussion.

So yeah, me too: -1 on proposal.

-- Keeping medicines from the bloodstreams of the sick; food from the bellies of the hungry; books from the hands of the uneducated; technology from the underdeveloped; and putting advocates of freedom in prisons. Intellectual property is to the 21st century what the slave trade was to the 16th.

9 Mar
9 Mar

12:31 p.m.

On 08/03/2014 18:49, Mark Dickinson wrote: > >

x = 49534541648432951

y = x + 2.0

x < y

True

Decimal(x) > Decimal(repr(y)) # simulating proposed meaning of Decimal(y)

True

You don't need to add 2.0 to x to see this effect. You can write y = float(x) or even y = x - 3.0 and you get the same value for y.

So in fact you can see

x < x - 3.0 True

which might be surprising at first sight.

[Just for amusement :-) ] Rob Cliffe

8 Mar
8 Mar

9:43 p.m.

Guido van Rossum guido@python.org wrote:

I wonder what Cowlishaw would say about our current discussion. He is also the father of Rexx...

It's in the second paragraph here:

http://speleotrove.com/decimal/daconvs.html

My reading is that he prefers exact conversions (if possible). This does not surprise me: Cowlishaw is a proponent of using all information from overlong inputs, which all functions of the specification do (he recently mentioned in a private mail that earlier versions of the specification did not have this feature and it was initially added for Java's BigDecimal -- he called it "an improvement").

Stefan Krah

11:20 p.m.

[Mark Dickinson]

Python's decimal module is based on Cowlishaw's standard, not on IEEE 754.

[Guido]

I wonder what Cowlishaw would say about our current discussion. He is also the father of Rexx...

I _expect_ Mike likes the status quo. Explaining why by analogy:

I'm certain the 754 designers would not like the status quo. To them string<->number conversions were "operations", no different in principle than the operations of, say, addition or root extraction. And _all_ operations in 754 treat the input(s) as infinitely precise, and produce an output correctly rounded according to the current context. Now that's technically not so for float->string operations in 754, but that's an irrelevant distraction (754 allowed for weaker rounding guarantees in that specific context because nobody knows how to achieve correct rounding efficiently in all cases in that context - and the members of the 754 committee later said they regretted allowing this exception).

Bottom line here: for all the 754 designers, and regardless of the type of x, Decimal(x) should accept x as exactly correct and produce a decimal object correctly rounded according to the current context (including setting context flags appropriately - e.g., signaling the inexact exception if any information in the input was lost due to rounding).

Now the specific instance of this Mike did pronounce on is Decimal(string). It's obvious that the 754 view is that the assumed-to-be infinitely precise string literal be rounded to the current context's precision (etc). But Mike (in email at the time) explicitly wanted to retain all the string digits in the returned decimal object, wholly ignoring the current context.

So that's what Python does. Is Decimal(0.1) really different? The 754 view, once you're used to it, is utterly predictable: Whatever the internal representation of 0.1, it's assumed to be infinitely precise, and you round its value to the decimal context's current precision. Mike's view is usually the same, but in one specific case of construction he thought differently. My guess is that he'd choose to be consistent with "the rule" for construction, having made an exception for it once already, than choose to be consistent with the other decimal operations.

Or we could hark back to REXX's desire to present an arithmetic that works the way people learned in school. I'd try that, except I'm not sure kids today learn arithmetic in school any more beyond learning how to push buttons ;-)

11:36 p.m.

On Sat, Mar 8, 2014 at 11:54 AM, Guido van Rossum guido@python.org wrote: >

On Sat, Mar 8, 2014 at 2:01 AM, Mark Dickinson dickinsm@gmail.com wrote: >

On Sat, Mar 8, 2014 at 9:43 AM, Mark Dickinson dickinsm@gmail.com wrote: > >

I see three sane options for float to Decimal conversion:

- Raise an exception.
- Round to the nearest Decimal value using the current context for that round operation.
- Do what we're currently doing, and do an exact conversion.

I think you're writing this entirely from the POV of an expert in floating point. And I'm glad we have experts like you around! I don't consider myself an expert at all, but I do think I have something to bring to the table -- insight the experience of non-expert users.

This reminded me a discussion we had with Mark Dickinson on the bug tracker: "When a user is entering 0.6112295, she means 0.6112295, not 0x1.38f312b1b36bdp-1 or 0.61122949999999998116351207499974407255649566650390625 which are exact values of the underlying binary representation of 0.6112295."

http://bugs.python.org/issue8860#msg108601

The topic was the conversion from binary floats to timedelta which is effectively a fixed point decimal with 6 decimal places after the point.

11:44 p.m.

On Saturday, March 8, 2014 5:36:18 PM UTC-6, Alexander Belopolsky wrote:

This reminded me a discussion we had with Mark Dickinson on the bug tracker: "When a user is entering 0.6112295, she means 0.6112295, not 0x1.38f312b1b36bdp-1 or 0.61122949999999998116351207499974407255649566650390625 which are exact values of the underlying binary representation of 0.6112295."

Yes, and I certainly don't want to start up the whole underlying discussion again, but very simply put, that's it !

There are two user issues:

1) maths should not give surprises

2) inputs should have a reasonable "expected" interpretation

marcus

9 Mar
9 Mar

12:06 a.m.

On Sat, Mar 8, 2014 at 6:44 PM, Mark H. Harris harrismh777@gmail.comwrote:

There are two user issues:

1) maths should not give surprises

2) inputs should have a reasonable "expected" interpretation

I am with Mark H. on this. While I appreciate the theoretical underpinnings of the status quo, I am yet to see data stored in binary that did not originate from decimal at some point. I think 99.999% of all instances of 0x1.199999999999ap+0 in numerical data come from a conversion of 1.1 from decimal to float rather than an a result of a computation that is correct to 16+ decimal places.

In a well-designed system, simple things should be simple and difficult things should be possible. Decimal(x) = Decimal(str(x)) is simple and what most users expect and ultimately need. Expert users will always have Decimal.from_float, contexts, round and other power tools at their disposal.

12:39 a.m.

On Sat, Mar 08, 2014 at 03:44:25PM -0800, Mark H. Harris wrote:

There are two user issues:

1) maths should not give surprises 2) inputs should have a reasonable "expected" interpretation

Computer maths cannot fail to give surprises.

py> x = 1/Decimal(3) py> sum([x, x, x]) == 1 False

"No surprises" is simply an impossible request.

-- Steven

1:58 a.m.

On Saturday, March 8, 2014 6:39:25 PM UTC-6, Steven D'Aprano wrote:

py> x = 1/Decimal(3) py> sum([x, x, x]) == 1 False

"No surprises" is simply an impossible request.

hi Steven, that's a straw man. Your argument presumes that False is a surprise. I am not in the least surprised. As, I am also not surprised by this:

dscale(32) 10100 d(1)/3 Decimal('0.33333333333333333333333333333333') <=== not a surprise, expected d(1)/3 + d(1)/3 +d(1)/3 Decimal('0.99999999999999999999999999999999') <=== not a surprise either, quite expected

And following:

dscale(42) 32 dSum= d(1)/3 + d(1)/3 +d(1)/3 dscale(38) 42 +dSum <===== at this point I expect to see the following line, just as I see it./ Decimal('1.0000000000000000000000000000000000000')

Cheers

2:46 a.m.

On Sun, Mar 9, 2014 at 12:58 PM, Mark H. Harris harrismh777@gmail.com wrote:

hi Steven, that's a straw man. Your argument presumes that False is a surprise. I am not in the least surprised. As, I am also not surprised by this:

dscale(32) 10100 d(1)/3 Decimal('0.33333333333333333333333333333333') <=== not a surprise, expected d(1)/3 + d(1)/3 +d(1)/3 Decimal('0.99999999999999999999999999999999') <=== not a surprise either, quite expected

I was trying to construct a proof from memory that would support this case, but I got tangled, so I cheated and went to Wikipedia for something similar.

Let's start with two variables, a and b, which we shall suppose are equal.

# Postulation? Or is this called an axiom? I don't remember.

a = b

# Wave hands vigorously and conclude that:

a = 0

Are you surprised? Probably. Normally, two variables being equal doesn't prove the value of either. Without knowing what actual handwaving went on in there, that is and should be a surprising result. Now, here's the handwaving, spelled out in full:

# As above, start here

a = b

# Multiply both sides by a

aa = ba

# Subtract bb (aka b squared, ASCII-friendly) from both sides

aa - bb = ba - bb

# Factor (both sides separately)

(a + b) (a - b) = b (a - b)

# Cancel out the multiplication by (a - b)

a + b = b

# Subtract b from both sides

a = 0

And there you are. To be unsurprised by the result, you have to know exactly what happened in between, and how it's different from the conventional rules of mathematics. Someone who already understands all that may well not be surprised, but someone who's expecting sane real-number arithmetic is in for a shock.

ChrisA

3:05 a.m.

On Saturday, March 8, 2014 8:46:21 PM UTC-6, Chris Angelico wrote:

# As above, start here

a = b

# Multiply both sides by a

aa = ba

# Subtract bb (aka b squared, ASCII-friendly) from both sides

aa - bb = ba - bb

# Factor (both sides separately)

(a + b) (a - b) = b (a - b)

# Cancel out the multiplication by (a - b) <======= if you divide by

zero, the universe will explode/

a + b = b

# Subtract b from both sides

a = 0

I found a proof once that proved that 1==2, but I can't find it... let you know.

Anyway (laughs) thanks for the grins

marcus

3:27 a.m.

On Saturday, March 8, 2014 8:46:21 PM UTC-6, Chris Angelico wrote: >

# As above, start here

a = b

# Multiply both sides by a

aa = ba

# Subtract bb (aka b squared, ASCII-friendly) from both sides

aa - bb = ba - bb

# Factor (both sides separately)

(a + b) (a - b) = b (a - b) <========of course all you really proved is that zero == zero

# Cancel out the multiplication by (a - b)

a + b = b

# Subtract b from both sides

a = 0

But then again, it really is a straw man (on steroids) because nobody is really surprised by the absurdity

postulate: a=b {magic} conclusion: a=0

At least no one should be surprised by an absurdity. On the other hand once they see the proof run by them, um, quickly--- slight of hand, as it were, they might be surprised if they don't catch it right away.

{still laughing}

love it

7:34 p.m.

I would dearly like to put this thread to rest, as it has strayed mightily from the topic of improvements to Python, and all points of view have been amply defended. I'm hoping to hear from Cowlishaw, but I expect he'll side with one of Mark Dickinson's proposals. I hope that somebody will take up the task to write a PEP about introducing a decimal literal; that sounds like an obtainable goal, less controversial than changing Decimal(<float>).

I did do some more thinking about how the magic repr() affects the distribution of values, and came up with an example of sorts that might show what it does. We've mostly focused on simple like 1.1, but to understand the distribution issue it's better to look at a very large value.

I took 2**49 as an example and added a random fraction. When printed this always gives a single digit past the decimal point, e.g. 562949953421312.5. Then I measured the distribution of the last digit. What I found matched my prediction: the digits 0, 1, 2, 4, 5, 6, 8, 9 occurred with roughly equal probability (1/8th). So 3 and 7 are completely missing.

The explanation is simple enough: using the (current) Decimal class it's easy to see that there are only 8 possible actual values, whose fractional part is a multiple of 1/8. IOW the exact values end in .000, .125, .250, .375, .500, .625, .750, .875. (*) The conclusion is that there are only 3 bits represented after the binary point, and repr() produces a single digit here, because that's all that's needed to correctly round back to the 8 possible values. So it picks the digit closest to each of the possible values, and when there are two possibilities it picks one. I don't know how it picks, but it is reproducible -- in this example it always chooses .2 to represent .250, and .8 to represent .750. The exact same thing happens later in the decimal expansion for smaller numbers.

I think that the main objection to this distribution is that, while the
exact distribution is evenly spaced (the possibilities are 1/8th away from
each other), the rounded distribution has some gaps of 1/10th and some of
1/5th. I am not enough of a statistician to know whether this matters (the
distribution of *digits* in the exact values is arguably less randomized
:-) but at least this example clarified my understanding of the phenomenon
we're talking about when we discuss distribution of values.

(*) I was momentarily startled to find that the set of Decimals produced contained 9 items, until I realized that some random() call must have produced a value close enough to 1 to be rounded up.

-- --Guido van Rossum (python.org/~guido)

8:07 p.m.

On 9 March 2014 19:34, Guido van Rossum guido@python.org wrote: >

I hope that somebody will take up the task to write a PEP about introducing a decimal literal; that sounds like an obtainable goal, less controversial than changing Decimal(<float>).

I said I'd have a go at this. It's not as simple as it might seem though.

The simplest proposal would be to say that 3.14d is equivalent to Decimal('3.14'). The fact the the constructor is independent of the context is good here. It means that the value of the literal can be determined statically which is good for trying to understand what code does and also for constant folding etc.

The problem though is with things like +3.14d or -3.14d. Python the language treats the + and - signs as not being part of the literal but as separate unary operators. This is unimportant for int, float and imaginary literals because there unary + is a no-op and unary - is exact. For decimal this is not the same as +Decimal('3.14') is not the same as Decimal('+3.14'):

```
>>> from decimal import Decimal, getcontext
>>> getcontext().prec = 3
>>> Decimal('+1234') # Exact
Decimal('1234')
>>> +Decimal('1234') # Rounded
Decimal('1.23E+3')
>>> Decimal('-1234') # Exact
Decimal('-1234')
>>> -Decimal('1234') # Rounded
Decimal('-1.23E+3')
```

I think that this behaviour would be surprising from a literal. I would prefer to be able to say that -1234d was equivalent to Decimal('-1234d') and would be guaranteed not to round. I just don't really know if that's trivially easy or really difficult given the current grammar specification and parse/compiler infrastructure.

Oscar

8:39 p.m.

Any solutions you are going to come up with would cause other anomalies such as -(5d) not being equal to -5d. My vote goes to treating + and - as the operators they are and telling people that if they want a negative constant that exceeds the context's precision they're going to have to use Decimal('-N'), rather than adding a terrible hack to the parser (and hence to the language spec).

BTW Do you why the default precision is 28?

On Sun, Mar 9, 2014 at 1:07 PM, Oscar Benjamin

oscar.j.benjamin@gmail.comwrote:

On 9 March 2014 19:34, Guido van Rossum guido@python.org wrote: >

I hope that somebody will take up the task to write a PEP about introducing a decimal literal; that sounds like an obtainable goal, less controversial than changing Decimal(<float>).

I said I'd have a go at this. It's not as simple as it might seem though.

The simplest proposal would be to say that 3.14d is equivalent to Decimal('3.14'). The fact the the constructor is independent of the context is good here. It means that the value of the literal can be determined statically which is good for trying to understand what code does and also for constant folding etc.

The problem though is with things like +3.14d or -3.14d. Python the language treats the + and - signs as not being part of the literal but as separate unary operators. This is unimportant for int, float and imaginary literals because there unary + is a no-op and unary - is exact. For decimal this is not the same as +Decimal('3.14') is not the same as Decimal('+3.14'):

```
>>> from decimal import Decimal, getcontext
>>> getcontext().prec = 3
>>> Decimal('+1234') # Exact
Decimal('1234')
>>> +Decimal('1234') # Rounded
Decimal('1.23E+3')
>>> Decimal('-1234') # Exact
Decimal('-1234')
>>> -Decimal('1234') # Rounded
Decimal('-1.23E+3')
```

I think that this behaviour would be surprising from a literal. I would prefer to be able to say that -1234d was equivalent to Decimal('-1234d') and would be guaranteed not to round. I just don't really know if that's trivially easy or really difficult given the current grammar specification and parse/compiler infrastructure.

Oscar

-- --Guido van Rossum (python.org/~guido)

10 Mar
10 Mar

10:07 a.m.

On 9 March 2014 20:39, Guido van Rossum guido@python.org wrote:

On Sun, Mar 9, 2014 at 1:07 PM, Oscar Benjamin oscar.j.benjamin@gmail.com wrote: >

The problem though is with things like +3.14d or -3.14d. Python the language treats the + and - signs as not being part of the literal but as separate unary operators. This is unimportant for int, float and imaginary literals because there unary + is a no-op and unary - is exact. For decimal this is not the same as +Decimal('3.14') is not the same as Decimal('+3.14'):

Any solutions you are going to come up with would cause other anomalies such as -(5d) not being equal to -5d. My vote goes to treating + and - as the operators they are and telling people that if they want a negative constant that exceeds the context's precision they're going to have to use Decimal('-N'), rather than adding a terrible hack to the parser (and hence to the language spec).

I think it's reasonable that -(5d) not be equal to -5d. Expert users would exploit this behaviour the same way that they currently exploit unary + for rounding to context. I don't think that non-expert users would write that by mistake. I'm also not that bothered about +5d rounding to context since the + is optional and current users of the decimal module would probably expect that to round.

What I dislike about this is that it would mean that negative decimal literals were essentially unsafe. It seems okay if you assume that the precision is 28 or higher but users can set it to a lower value and decimal contexts are unscoped so they have action at a distance on other code. So if I have a module libmod.py with:

```
# libmod.py
def libfunc():
a = -1.23465789d
return f(a)
```

Then I have a script:

```
# script.py
from decimal import localcontext
import libmod
with localcontext() as ctx:
ctx.prec = 5
b = libmod.libfunc()
```

I think we could easily get into a situation where the author of libmod just assumes that the decimal context has enough precision and the author of script.py doesn't realise the effect that their change of context has on libmod.

These kinds of things are generally problematic when using the decimal module and the solution is typically that libfunc needs to take control of the context. I think that it would be particularly surprising with decimal literals though. The change in the precision of "a" above might be a harmless reduction in the precision of the result or it could lead to an infinite loop or basically anything else depending on what libmod uses the value for.

BTW Do you why the default precision is 28?

I have no idea. The standards I've read describe a few particular 9 digit contexts that should be provided by a conforming implementation and decimal provides them but neither is the default:

import decimal decimal.BasicContext.prec 9 decimal.ExtendedContext.prec 9

AFAIK there's no standard that suggests 28 digits. There are other common contexts such as decimal32 (7 digits), decimal64 (16 digits), and decimal128 (34 digits) and these are the contexts that Java's BigDecimal provides (as well as an "unlimited" context).

Oscar

10:59 a.m.

On Mon, Mar 10, 2014 at 10:07:11AM +0000, Oscar Benjamin wrote:

On 9 March 2014 20:39, Guido van Rossum guido@python.org wrote:

On Sun, Mar 9, 2014 at 1:07 PM, Oscar Benjamin oscar.j.benjamin@gmail.com wrote: >

The problem though is with things like +3.14d or -3.14d. Python the language treats the + and - signs as not being part of the literal but as separate unary operators.

Is that still true? Possibly the peephole optimizer has changed the situation?

Using Python 1.5:

from dis import dis dis(compile("-5", "", "single")) 0 SET_LINENO 0

```
3 SET_LINENO 1
6 LOAD_CONST 0 (5)
9 UNARY_NEGATIVE
10 PRINT_EXPR
11 LOAD_CONST 1 (None)
14 RETURN_VALUE
```

but using Python 2.4:

py> dis(compile("-5", "", "single")) 1 0 LOAD_CONST 0 (-5) 3 PRINT_EXPR 4 LOAD_CONST 1 (None) 7 RETURN_VALUE

(versions more recent than 2.4 also use a constant).

This is unimportant for int, float and imaginary literals because there unary + is a no-op and unary - is exact. For decimal this is not the same as +Decimal('3.14') is not the same as Decimal('+3.14'):

Any solutions you are going to come up with would cause other anomalies such as -(5d) not being equal to -5d. My vote goes to treating + and - as the operators they are and telling people that if they want a negative constant that exceeds the context's precision they're going to have to use Decimal('-N'), rather than adding a terrible hack to the parser (and hence to the language spec).

I think it's reasonable that -(5d) not be equal to -5d.

Did you leave the word "not" out of that sentence? :-)

I don't think that it is reasonable for -(5d) to not equal -5d. I think
that's an awful interface, and terribly surprising. I don't think that
there are any situations were an otherwise redundant pair of parens
changes behavious. E.g. x*(y) is the same as x*y.

(Parens are sometimes used to disambiguate syntax or change precedence. That's different, since the parens aren't redundant.)

Considering that the motivation for this change is to make it easier for newbies and numerically naive users, I really do not relish having to explain to newbies why -5d and -(5d) are different.

-- Steven

12:12 p.m.

On 10 March 2014 10:59, Steven D'Aprano steve@pearwood.info wrote:

On Mon, Mar 10, 2014 at 10:07:11AM +0000, Oscar Benjamin wrote:

On 9 March 2014 20:39, Guido van Rossum guido@python.org wrote:

On Sun, Mar 9, 2014 at 1:07 PM, Oscar Benjamin oscar.j.benjamin@gmail.com wrote: >

The problem though is with things like +3.14d or -3.14d. Python the language treats the + and - signs as not being part of the literal but as separate unary operators.

Is that still true? Possibly the peephole optimizer has changed the situation?

Yes it does. It also does the same for "complex literals" even though the language formally only defines imaginary literals. The question is how to define exactly the grammar. Then once defined if the result is independent of any context the optimiser can safely constant fold decimal literals in the same way.

The grammar is here: http://docs.python.org/3.4/reference/grammar.html

The relevant part is here:

factor: ('+'|'-'|'~') factor | power power: atom trailer* ['**' factor] atom: ('(' [yield_expr|testlist_comp] ')' | '[' [testlist_comp] ']' | '{' [dictorsetmaker] '}' | NAME | NUMBER | STRING+ | '...' | 'None' | 'True' | 'False')

So if the specification is that decimal literals go into the NUMBER
part just like int, float and imaginary literals do then the + or -
are at a separate place. The expression -5 ** 2 is evaluated as -(5 **
2) which is -25 rather than (-5) ** 2 which is +25. It may be possible
but it's certainly not straight-forward to rewrite the grammar so that
-25d becomes D('-25') without affecting the normal precedence rules.

I think it's reasonable that -(5d) not be equal to -5d.

Did you leave the word "not" out of that sentence? :-)

It's reasonable if you're an experienced user of the decimal module but yeah okay it's not really reasonable in a general sense...

I don't think that it is reasonable for -(5d) to not
equal -5d. I think
that's an awful interface, and terribly surprising. I don't think that
there are any situations were an otherwise redundant pair of parens
changes behavious. E.g. x*(y) is the same as x*y.

(Parens are sometimes used to disambiguate syntax or change precedence. That's different, since the parens aren't redundant.)

Considering that the motivation for this change is to make it easier for newbies and numerically naive users, I really do not relish having to explain to newbies why -5d and -(5d) are different.

Yes but newbies won't write -(5d). Experts might if it had the semantics they want. The current behaviour of the Decimal type is already surprising in this regard so unless __pos__ and __neg__ are changed there will always be cases that surprise people.

Oscar

1:38 p.m.

On Mon, Mar 10, 2014 at 12:12 PM, Oscar Benjamin <oscar.j.benjamin@gmail.com

wrote:

On 10 March 2014 10:59, Steven D'Aprano steve@pearwood.info wrote:

On Mon, Mar 10, 2014 at 10:07:11AM +0000, Oscar Benjamin wrote:

On 9 March 2014 20:39, Guido van Rossum guido@python.org wrote:

On Sun, Mar 9, 2014 at 1:07 PM, Oscar Benjamin oscar.j.benjamin@gmail.com wrote: >

The problem though is with things like +3.14d or -3.14d. Python the language treats the + and - signs as not being part of the literal but as separate unary operators.

Is that still true? Possibly the peephole optimizer has changed the situation?

Yes it does. It also does the same for "complex literals" even though the language formally only defines imaginary literals.

... and don't forget the Python 2.x-only hack for negation of integers:

type(-9223372036854775808)

<type 'int'>

type(-(9223372036854775808))

<type 'long'>

which means that it's not true that Python 2.x behaves "as if" there were
no negative literals. Python 3 is cleaner in this respect. I guess this
shows that we *could* in theory reintroduce such a hack for negation of
decimal literals, but I agree with everyone else so far that that would be
a bad idea.

-- Mark

12:57 p.m.

On 10 Mar 2014 21:06, "Steven D'Aprano" steve@pearwood.info wrote: >

On Mon, Mar 10, 2014 at 10:07:11AM +0000, Oscar Benjamin wrote:

On 9 March 2014 20:39, Guido van Rossum guido@python.org wrote:

On Sun, Mar 9, 2014 at 1:07 PM, Oscar Benjamin oscar.j.benjamin@gmail.com wrote: >

Is that still true? Possibly the peephole optimizer has changed the situation?

Using Python 1.5:

from dis import dis dis(compile("-5", "", "single")) 0 SET_LINENO 0

```
3 SET_LINENO 1
6 LOAD_CONST 0 (5)
9 UNARY_NEGATIVE
10 PRINT_EXPR
11 LOAD_CONST 1 (None)
14 RETURN_VALUE
```

but using Python 2.4:

py> dis(compile("-5", "", "single")) 1 0 LOAD_CONST 0 (-5) 3 PRINT_EXPR 4 LOAD_CONST 1 (None) 7 RETURN_VALUE

(versions more recent than 2.4 also use a constant).

It actually makes the discrepancy worse, rather than better - the optimiser
is just doing constant folding at compile time, so it would just be
rounding using the context that applied at compile time. That context
dependence creates problems for the entire *notion* of constant folding
decimal literals - you can't constant fold them at compile time at all,
because the context may be wrong.

I think users of decimal literals will just need to deal with the risk of unexpected rounding, as the alternatives are even more problematic.

Cheers, Nick.

> >

This is unimportant for int, float and imaginary literals because there unary + is a no-op and unary - is exact. For decimal this is not the same as +Decimal('3.14') is not the same as Decimal('+3.14'):

I think it's reasonable that -(5d) not be equal to -5d.

Did you leave the word "not" out of that sentence? :-)

I don't think that it is reasonable for -(5d) to not equal -5d. I think
that's an awful interface, and terribly surprising. I don't think that
there are any situations were an otherwise redundant pair of parens
changes behavious. E.g. x*(y) is the same as x*y.

(Parens are sometimes used to disambiguate syntax or change precedence. That's different, since the parens aren't redundant.)

Considering that the motivation for this change is to make it easier for newbies and numerically naive users, I really do not relish having to explain to newbies why -5d and -(5d) are different.

-- Steven

Python-ideas mailing list Python-ideas@python.org https://mail.python.org/mailman/listinfo/python-ideas Code of Conduct: http://python.org/psf/codeofconduct/

1:53 p.m.

[My apologies for being terse, I don't have much time to follow this discussion right now.]

Nick Coghlan ncoghlan@gmail.com wrote:

I think users of decimal literals will just need to deal with the risk of unexpected rounding, as the alternatives are even more problematic.

That is why I think we should seriously consider moving to IEEE semantics for a decimal literal. Among other things:

Always round the literal inputs.

Supply IEEE contexts.

Make Decimal64 the default.

Add the missing rounding modes to sqrt() and exp().

Keep the ability to create exact Decimals through the constructor when no context is passed.

Make the Decimal constructor use the context for rounding if it is passed.

...

The existing specification is largely compatible with IEEE 754-2008:

http://speleotrove.com/decimal/dascope.html

We can still support setting irregular (non IEEE) contexts.

Stefan Krah

2:05 p.m.

On 10 March 2014 13:53, Stefan Krah stefan@bytereef.org wrote:

[My apologies for being terse, I don't have much time to follow this discussion right now.]

Nick Coghlan ncoghlan@gmail.com wrote:

I think users of decimal literals will just need to deal with the risk of unexpected rounding, as the alternatives are even more problematic.

That is why I think we should seriously consider moving to IEEE semantics for a decimal literal. Among other things:

Always round the literal inputs.

Supply IEEE contexts.

Make Decimal64 the default.

Add the missing rounding modes to sqrt() and exp().

Keep the ability to create exact Decimals through the constructor when no context is passed.

Make the Decimal constructor use the context for rounding if it is passed.

...

The existing specification is largely compatible with IEEE 754-2008:

http://speleotrove.com/decimal/dascope.html

We can still support setting irregular (non IEEE) contexts.

This is what I've been thinking about. Most non-expert users will be very happy with Decimal64 and a single fixed context. There could be a separate type called decimal64 in builtins that always used the same standard context. Literals could create decimal64 instances.

The semantics of code that uses this type and decimal literals would be independent of any arithmetic context which is good not just for constant folding but for understanding. There would be no need to explain what an arithmetic context is to new users. You can just say: "here's a type that represents decimal values with 16 digits. It sometimes needs to round if the result of a calculation exceeds 16 digits so it uses the standard decimal rounding mode XXX."

Where this would get complicated is for people who also use the Decimal type. They'd need to keep track of which objects were of which type and so decimal literals might seem more annoying than useful.

Oscar

3:31 p.m.

On Mon, Mar 10, 2014 at 02:05:22PM +0000, Oscar Benjamin wrote:

This is what I've been thinking about. Most non-expert users will be very happy with Decimal64 and a single fixed context. There could be a separate type called decimal64 in builtins that always used the same standard context. Literals could create decimal64 instances.

The semantics of code that uses this type and decimal literals would be independent of any arithmetic context which is good not just for constant folding but for understanding. There would be no need to explain what an arithmetic context is to new users. You can just say: "here's a type that represents decimal values with 16 digits. It sometimes needs to round if the result of a calculation exceeds 16 digits so it uses the standard decimal rounding mode XXX."

All this sounds quite good to me. What's the catch? :-)

Where this would get complicated is for people who also use the Decimal type. They'd need to keep track of which objects were of which type and so decimal literals might seem more annoying than useful.

Hmmm. I don't think this should necessarily be complicated, or at least no more complicated than dealing with any other mixed numeric types. If you don't want to mix them, don't mix them :-)

(Perhaps there could be a Decimal context flag to allow/disallow mixed Decimal/decimal64 operations?)

I would expect that decimal64 and decimal.Decimal would be separate types. They might share parts of the implementation under the hood, but I don't think it would be necessary or useful to have decimal64 inherit from Decimal, or visa versa.

-- Steven

4 p.m.

On 10 March 2014 15:31, Steven D'Aprano steve@pearwood.info wrote:

On Mon, Mar 10, 2014 at 02:05:22PM +0000, Oscar Benjamin wrote:

This is what I've been thinking about. Most non-expert users will be very happy with Decimal64 and a single fixed context. There could be a separate type called decimal64 in builtins that always used the same standard context. Literals could create decimal64 instances.

Where this would get complicated is for people who also use the Decimal type. They'd need to keep track of which objects were of which type and so decimal literals might seem more annoying than useful.

Hmmm. I don't think this should necessarily be complicated, or at least no more complicated than dealing with any other mixed numeric types. If you don't want to mix them, don't mix them :-)

Exactly. It just means that you wouldn't want to use the literals in code that actually uses the decimal module. Consider:

if some_condition: x = 1d else: x = Decimal(some_string)

# ...

y = x / 3

So now x / 3 rounds differently depending on whether x is a decimal64 or a Decimal. I probably don't want that. The solution: coerce to Decimal. But then why did I bother with the Decimal literal anyway? Decimal(1d) is hardly better than Decimal('1').

Oscar

11 Mar
11 Mar

1:17 a.m.

On Mon, Mar 10, 2014 at 04:00:38PM +0000, Oscar Benjamin wrote:

On 10 March 2014 15:31, Steven D'Aprano steve@pearwood.info wrote:

On Mon, Mar 10, 2014 at 02:05:22PM +0000, Oscar Benjamin wrote:

Hmmm. I don't think this should necessarily be complicated, or at least no more complicated than dealing with any other mixed numeric types. If you don't want to mix them, don't mix them :-)

Exactly. It just means that you wouldn't want to use the literals in code that actually uses the decimal module. Consider:

If you're writing a function that accepts whatever random numeric type the caller passes in, you have to expect that you cannot control the behaviour precisely. (You can't control rounding or precision for floats; Fractions are exact; the proposed decimal64 will have a fixed precision and fixed rounding; decimal.Decimal you can control.) You might still write type-agnostic code, but the result you get may depend on the input data.

if some_condition: x = 1d else: x = Decimal(some_string)

Mixing different types within a single calculation/function, as shown above, sounds like a recipe for chaos to me. I wouldn't write it like that, any more than I would write:

if some_condition: x = 1.1 else: x = Decimal("2.2")

# ...

y = x / 3

So now x / 3 rounds differently depending on whether x is a decimal64 or a Decimal. I probably don't want that.

Then don't do it! :-)

The solution: coerce to Decimal.

Or write both branches as a Decimal in the first place.

But then why did I bother with the Decimal literal anyway? Decimal(1d) is hardly better than Decimal('1').

Presumably you didn't use a decimal64 literal if you cared about controlling the rounding or precision, because that would be silly. (You might, on the other hand, use decimal64 literals everywhere in your function, if its rounding and precision was sufficient for your needs.) However, the caller of your function may pass you a decimal64 instance. I don't think there's much advantage to passing integer values with the d suffix, but passing fractional values is a different story:

some_function(1.1)

versus

some_function(1.1d)

I think this is a proposal I could get behind: leave the decimal module (mostly) as-is, perhaps with a few backwards-compatible tweaks, while adding an independent decimal64 or decimal128 built-in type and literal with fixed rounding and precision. The new literal type would satisfy the use-case Mark Harris is worried about, apart from having to add a "d" suffix to literals[1], while the decimal module still allows for fine control of the context.

The built-in decimal type should convert floats using their repr, which ought to make Guido happy too :-)

[1] Shifting to decimal floats by default is a major backwards-incompatible change.

-- Steven

10 Mar
10 Mar

2:22 p.m.

On Mon, Mar 10, 2014 at 1:53 PM, Stefan Krah stefan@bytereef.org wrote:

That is why I think we should seriously consider moving to IEEE semantics for a decimal literal.

I think that would make a *lot* of sense.

-- Mark

3:01 p.m.

On Mon, Mar 10, 2014 at 7:22 AM, Mark Dickinson dickinsm@gmail.com wrote:

On Mon, Mar 10, 2014 at 1:53 PM, Stefan Krah stefan@bytereef.org wrote:

That is why I think we should seriously consider moving to IEEE semantics

for a decimal literal.

I think that would make a *lot* of sense.

What's the proposal? Just using decimal64 for decimal literals, or introducing decimal64 as a new builtin type? (I could get behind either one.)

-- --Guido van Rossum (python.org/~guido)

6:09 p.m.

Guido van Rossum guido@python.org wrote:

On Mon, Mar 10, 2014 at 7:22 AM, Mark Dickinson dickinsm@gmail.com wrote: On Mon, Mar 10, 2014 at 1:53 PM, Stefan Krah stefan@bytereef.org wrote:

```
That is why I think we should seriously consider moving to IEEE
semantics
for a decimal literal.
I think that would make a *lot* of sense.
```

What's the proposal? Just using decimal64 for decimal literals, or introducing decimal64 as a new builtin type? (I could get behind either one.)

IEEE 754-2008 is in a certain sense "arbitrary precision", since it allows multiple discrete contexts: Decimal32, Decimal64, Decimal128, ...

In theory this goes on ad infinitum, but the precision increases much slower than the exponents. Mike Cowlishaw's spec fills in the gaps by allowing arbitrary contexts. Additionally there are some minor differences, but if we make moderate changes to Decimal, we get strict IEEE behavior.

So my specific proposal was:

1) Make those changes to Decimal (we can call the module decimal2
if backwards compatibility rules it out). The most important
change relevant to this subthread is rounding all *literals* on
input while preserving exact construction via Decimal(value).

2) Keep the arbitrary context facility.

3) Set IEEEContext(Decimal64) as the default. Users will most likely primarily use Decimal32, Decimal64 and Decimal128. (Many users will likely never change the context at all.)

4) Optional: Also use the function names from IEEE 754-2008.

With these changes most users would /think/ that Decimal is a fixed width Decimal64 type. Advanced users can still change the context. I know for a fact that some users really like the option of increasing the precision temporarily.

I have to think about the other solution (decimal64 only). At first glance it seems too restrictive, since I imagine users would at least want Decimal128, too. Additionally there is no speed benefit.

Stefan Krah

8:51 p.m.

On 10 March 2014 18:09, Stefan Krah stefan@bytereef.org wrote:

Guido van Rossum guido@python.org wrote:

On Mon, Mar 10, 2014 at 7:22 AM, Mark Dickinson dickinsm@gmail.com wrote: On Mon, Mar 10, 2014 at 1:53 PM, Stefan Krah stefan@bytereef.org wrote:

```
That is why I think we should seriously consider moving to IEEE
semantics
for a decimal literal.
I think that would make a *lot* of sense.
```

What's the proposal? Just using decimal64 for decimal literals, or introducing decimal64 as a new builtin type? (I could get behind either one.)

IEEE 754-2008 is in a certain sense "arbitrary precision", since it allows multiple discrete contexts: Decimal32, Decimal64, Decimal128, ...

In theory this goes on ad infinitum, but the precision increases much slower than the exponents. Mike Cowlishaw's spec fills in the gaps by allowing arbitrary contexts. Additionally there are some minor differences, but if we make moderate changes to Decimal, we get strict IEEE behavior.

So my specific proposal was:

1) Make those changes to Decimal (we can call the module decimal2
if backwards compatibility rules it out). The most important
change relevant to this subthread is rounding all *literals* on
input while preserving exact construction via Decimal(value).

2) Keep the arbitrary context facility.

3) Set IEEEContext(Decimal64) as the default. Users will most likely primarily use Decimal32, Decimal64 and Decimal128. (Many users will likely never change the context at all.)

4) Optional: Also use the function names from IEEE 754-2008.

I generally agree with the above except that...

With these changes most users would /think/ that Decimal is a fixed width Decimal64 type. Advanced users can still change the context.

This is the problem I have with this particular proposal. Users would think that it's a fixed-width type and then write code that naively makes that assumption. Then the code blows up when someone else changes the arithmetic context. I don't think we should encourage non-expert users to think that they can safely rely on this behaviour without actually making it safe to rely on.

I know for a fact that some users really like the option of increasing the precision temporarily.

Agreed. I do this often. But I think it's the kind of thing that happens in a special library like Mark's pcdeclib rather than application code for a casual user. The option will always be there to promote your decimal64s to the Big Decimal type and do all your calculations in whichever precision you want.

I have to think about the other solution (decimal64 only). At first glance it seems too restrictive, since I imagine users would at least want Decimal128, too. Additionally there is no speed benefit.

Decimal128 seems fine to me. I just think it should be a true fixed-width type. The benefits of this are: 1) Naive code doesn't get broken by different contexts. 2) I can tell by looking at a snippet exactly what it does without needing to wonder (or ask) whether or not the context has been fiddled with. 3) I can show code that uses decimal literals and the decimal128 constructor and guarantee that it works without caveats. 4) Since the meaning of any expression is known at compile time it is amenable to constant folding. 5) Unary + is a no-op and - is exact (as users would expect) so negative literals will have the same meaning regardless of the current context.

Oscar

11:02 p.m.

On 11 Mar 2014 01:02, "Guido van Rossum" guido@python.org wrote: >

On Mon, Mar 10, 2014 at 7:22 AM, Mark Dickinson dickinsm@gmail.com wrote: >

On Mon, Mar 10, 2014 at 1:53 PM, Stefan Krah stefan@bytereef.org wrote:

That is why I think we should seriously consider moving to IEEE semantics for a decimal literal.

I think that would make a *lot* of sense.

My take is that we're down to two main options:

Stefan: use the existing decimal type, change default context to Decimal64, round all decimal literals to current context

Oscar: new fixed width decimal64 builtin type, always uses Decimal64 context (allowing constant folding), interoperates with variable context decimal.Decimal values (producing a decimal.Decimal result)

I lean towards Oscar's proposal, as it removes the hidden context dependent behaviour and makes the builtin decimals true constant values.

Perhaps Stefan & Oscar would be willing to collaborate on a PEP that sums up the pros and cons of the two main alternatives (as well as some of the other variants that were discussed and rejected)?

(Adding the IEEE contexts to the decimal module sounds like a good idea regardless)

Cheers, Nick.

>

-- --Guido van Rossum (python.org/~guido)

Python-ideas mailing list Python-ideas@python.org https://mail.python.org/mailman/listinfo/python-ideas Code of Conduct: http://python.org/psf/codeofconduct/

11 Mar
11 Mar

6:15 a.m.

On 3/10/2014 7:02 PM, Nick Coghlan wrote:

My take is that we're down to two main options:

Stefan: use the existing decimal type, change default context to Decimal64, round all decimal literals to current context

Oscar: new fixed width decimal64 builtin type, always uses Decimal64 context (allowing constant folding), interoperates with variable context decimal.Decimal values (producing a decimal.Decimal result)

I lean towards Oscar's proposal, as it removes the hidden context dependent behaviour and makes the builtin decimals true constant values.

Ditto for me. An everyday replacement for binary floats should be about as simple as binary floats, and simpler in the sense of having less surprising behavior.

-- Terry Jan Reedy

7:24 a.m.

On Tue, 11 Mar 2014 09:02:31 +1000 Nick Coghlan ncoghlan@gmail.com wrote:

My take is that we're down to two main options:

Stefan: use the existing decimal type, change default context to Decimal64, round all decimal literals to current context

Oscar: new fixed width decimal64 builtin type, always uses Decimal64 context (allowing constant folding), interoperates with variable context decimal.Decimal values (producing a decimal.Decimal result)

I lean towards Oscar's proposal, as it removes the hidden context dependent behaviour and makes the builtin decimals true constant values.

Yuck. Is this some kind of joke? We've gone through the trouble of unifying long and int in Python 3 and now people are proposing two different Decimal types, one with fixed precision and one with arbitrary precision?

I'm completely -1 on this.

Regards

Antoine.

10:51 a.m.

On 11 March 2014 07:24, Antoine Pitrou solipsis@pitrou.net wrote:

On Tue, 11 Mar 2014 09:02:31 +1000 Nick Coghlan ncoghlan@gmail.com wrote: >

My take is that we're down to two main options:

Yuck. Is this some kind of joke? We've gone through the trouble of unifying long and int in Python 3 and now people are proposing two different Decimal types, one with fixed precision and one with arbitrary precision?

I'm completely -1 on this.

I understand your objection Antoine. There is a big difference though
between integer and non-integer types in this respect though. For
integers is is not hard to define a single integer type that satisfies
the vast majority of use-cases (focusing purely on semantics rather
than detailed implementation aspects and performance). The only
contentious issue is how to handle inexact division. int and long were
not really "unified" in Python 3. The long type *replaced* the int
type as it has those universal integer semantics that are really
wanted.

When it comes to non-integer types there's just no single good way of
doing it. So just in the stdlib we already have float, Fraction and
Decimal. The additional libraries that I use have many more numeric
types than that. The idea here is that for most users decimal128 just
is *the* decimal type. The decimal.Decimal type is IMO overly complex
and should really be for niche use-cases like other non-stdlib
multi-precision numeric types.

Java's BigDecimal is more user friendly in some important ways. It performs all calculations exactly until you explicitly round. When you ask it to do something that must be inexact without explicitly supplying a rounding context it throws an exception. Naturally it is more verbose than calculations with decimal.Decimal but it is possible to always know what's going on just by looking at the relevant code.

C# has a (non-standard I think) fixed-width decimal type. Once again the precision and exponent range of this type are not governed by a hidden modifiable arithmetic context so it is possible to know what a snippet of code does before run-time.

C++ is introducing decimal32, decimal64, and decimal128 as fixed-width decimal floating point types as according to IEEE-754-2008. These will be fixed-width types whose precision is always known so that a static analysis can explain what code does (excepting traps and flags - I'm not clear on how they will work in C++).

The problem as I see it is that if Decimal is brought into the core of the language in its current format then many people will be confused by its behaviour. The behaviour of Python's Decimal type is governed by the hidden global variable that is the arithmetic context so that even trivial looking code cannot fully be understood without thinking through all the possible contexts that might be in effect when it is executed. I've used a few multi-precision libraries and I find Decimal hard to use because of the action at a distance effect of the contexts and the fact that it is possible for Decimals to exist whose precision exceeds the context so that even the expression -x is subject to rounding.

The idea is here is to make decimal floating point accessible for non-expert users. I'm concerned that the current Decimal type is overly complex for this use-case.

Oscar

11:46 a.m.

On 11 March 2014 20:51, Oscar Benjamin oscar.j.benjamin@gmail.com wrote:

On 11 March 2014 07:24, Antoine Pitrou solipsis@pitrou.net wrote:

On Tue, 11 Mar 2014 09:02:31 +1000 Nick Coghlan ncoghlan@gmail.com wrote: >

My take is that we're down to two main options:

Yuck. Is this some kind of joke? We've gone through the trouble of unifying long and int in Python 3 and now people are proposing two different Decimal types, one with fixed precision and one with arbitrary precision?

I'm completely -1 on this.

I understand your objection Antoine. There is a big difference though
between integer and non-integer types in this respect though. For
integers is is not hard to define a single integer type that satisfies
the vast majority of use-cases (focusing purely on semantics rather
than detailed implementation aspects and performance). The only
contentious issue is how to handle inexact division. int and long were
not really "unified" in Python 3. The long type *replaced* the int
type as it has those universal integer semantics that are really
wanted.

When it comes to non-integer types there's just no single good way of
doing it. So just in the stdlib we already have float, Fraction and
Decimal. The additional libraries that I use have many more numeric
types than that. The idea here is that for most users decimal128 just
is *the* decimal type. The decimal.Decimal type is IMO overly complex
and should really be for niche use-cases like other non-stdlib
multi-precision numeric types.

Right. I believe the appropriate comparison here should be with builtin floats (which are essentially 64 bit C doubles, with all the limitations that implies), rather than with the historical int/long situation.

The reason I see Oscar's proposal as arguably different from the
int/long case is that we *wouldn't* be trying to pretend that builtin
decimals are the same type as the existing variable context decimals.
While they would implicitly interoperate under addition, etc (as
numbers generally do, with variable precision decimals being treated
as the resulting type for mixed arithmetic), there would be no way to
implicitly turn two fixed precision decimals into variable precision
ones.

That is, don't think "int vs long", think "float16 vs float32 vs
float64 vs float128", with Oscar's proposal being that picking just
one decimal floating point precision as *the* floating point
precision, just as Python already does for binary floating point,
makes more sense than trying to provide variable precision support in
a builtin type.

That approach buys us a couple of key benefits:

- they would be true constants, so constant folding, etc, would work normally, and you could tell from reading the code exactly what it will do, without needing to worry about the impact of thread local state
- you could generally use the builtin decimal literals without learning anything about decimal contexts, because decimal literals wouldn't be context dependent (although their behaviour would be formally defined in terms of an IEEE decimal context, for most users it would just be "this is how Python decimal literals behave")

The guidance to new users would then be *don't* use the decimal
module, use decimal literals instead. If you need more configurable
behaviour, *then* reach for the decimal module. However, the explicit
methods on decimal context objects should probably still be updated to
accept both fixed and variable precision decimals under this model.

Cheers, Nick.

-- Nick Coghlan | ncoghlan@gmail.com | Brisbane, Australia

12:33 p.m.

On 11.03.2014 12:46, Nick Coghlan wrote:

...[Oscar's proposal being that picking just one
decimal floating
point precision as *the* floating point precision]...

That approach buys us a couple of key benefits:

- they would be true constants, so constant folding, etc, would work normally, and you could tell from reading the code exactly what it will do, without needing to worry about the impact of thread local state
- you could generally use the builtin decimal literals without learning anything about decimal contexts, because decimal literals wouldn't be context dependent (although their behaviour would be formally defined in terms of an IEEE decimal context, for most users it would just be "this is how Python decimal literals behave")

The guidance to new users would then be *don't* use the decimal
module, use decimal literals instead. If you need more configurable
behaviour, *then* reach for the decimal module. However, the explicit
methods on decimal context objects should probably still be updated to
accept both fixed and variable precision decimals under this model.

I think you are leaving out one of the most important use cases for decimal values: data input and output.

The literals would only appear in programs. However, in most use cases, you want to read decimal data from some source, process it and then write it back again. This would most likely also require using a few decimal literals, but the main source of decimals would still be constructors that you choose when writing the tools.

Coming back to floats and the decimal constructor:

In investment banking, for example, people usually work with floats all the time. You only convert back to decimals at the very end of some calculation. The reason here being either that the calculations involve complex operations which are not available for decimals, trying to keep error intervals small, or a combination of both. In general, you try to use as much precision as you can afford (in terms of memory and speed), to keep those error intervals small.

In accounting, you often use decimals for storing data with an (usually contractually or law based) agreed upon precision and rounding logic. However, there situations where you have to go to floats as well in order to run calculations, e.g. for interest, taxes, etc.

In both situations, you want to have the decimal constructor take the float values with full precision and only then apply the necessary rounding to turn the value into a form which complies with the agreed upon rules. It's also not uncommon to add correctional bookings to address rounding issues explicitly (e.g. when calculating VAT of a large number of individual items).

In short: you try to prevent rounding from happening as much as possible and when using it, you use it in a controlled way.

Based on this, the choice to have the decimal constructor use full precision when reading floats is a good one, even though it may not feel right for the novice, the casual decimal user or as human concept :-)

For decimal literals, I'd argue that if you enter a value 1.2500d, you are expecting a decimal with 4 decimal places precision, not 64 or 128 bits :-)

The information about the intended precision is implicit in the literal. You see this done in exchange rates, stock prices, prices at your grocery store, etc.

The decimals can then be transformed into ones with higher precision during calculations and then possibly back to lower precision, but this is an explicit decision by the system doing the calculation.

Anyway, just throwing in some additional entropy into this discussion. Probably not all that helpful, since you're already converging on two possible solutions :-)

-- Marc-Andre Lemburg eGenix.com

Professional Python Services directly from the Source (#1, Mar 11 2014)

Python Projects, Consulting and Support ... http://www.egenix.com/ mxODBC.Zope/Plone.Database.Adapter ... http://zope.egenix.com/ mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/

2014-04-09: PyCon 2014, Montreal, Canada ... 29 days to go

::::: Try our mxODBC.Connect Python Database Interface for free ! ::::::

eGenix.com Software, Skills and Services GmbH Pastor-Loeh-Str.48 D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg Registered at Amtsgericht Duesseldorf: HRB 46611 http://www.egenix.com/company/contact/

1:04 p.m.

On 11 March 2014 12:33, M.-A. Lemburg mal@egenix.com wrote:

On 11.03.2014 12:46, Nick Coghlan wrote: >

...[Oscar's proposal being that picking just one
decimal floating
point precision as *the* floating point precision]...

I think you are leaving out one of the most important use cases for decimal values: data input and output.

The literals would only appear in programs. However, in most use cases, you want to read decimal data from some source, process it and then write it back again. This would most likely also require using a few decimal literals, but the main source of decimals would still be constructors that you choose when writing the tools.

True, and the Decimal constructor does what you want. The decimal128 constructor will also do what you want provided you don't need more than 34 digits. In my proposal you would still be able to trap Inexact with decimal128 if desired.

Coming back to floats and the decimal constructor:

In investment banking, for example, people usually work with floats all the time. You only convert back to decimals at the very end of some calculation. The reason here being either that the calculations involve complex operations which are not available for decimals, trying to keep error intervals small, or a combination of both. In general, you try to use as much precision as you can afford (in terms of memory and speed), to keep those error intervals small.

In accounting, you often use decimals for storing data with an (usually contractually or law based) agreed upon precision and rounding logic.

Well this is a situation where you would definitely want all of the bells and whistles of the full Decimal module. You could still use decimal literals in this code and they would inter-operate as expected.

However, there situations where you have to go to floats as well in order to run calculations, e.g. for interest, taxes, etc.

In both situations, you want to have the decimal constructor take the float values with full precision and only then apply the necessary rounding to turn the value into a form which complies with the agreed upon rules. It's also not uncommon to add correctional bookings to address rounding issues explicitly (e.g. when calculating VAT of a large number of individual items).

In short: you try to prevent rounding from happening as much as possible and when using it, you use it in a controlled way.

Based on this, the choice to have the decimal constructor use full precision when reading floats is a good one, even though it may not feel right for the novice, the casual decimal user or as human concept :-)

Don't worry, I think that the suggestion to change the way that Decimal(float) works is settled now.

For decimal literals, I'd argue that if you enter a value 1.2500d, you are expecting a decimal with 4 decimal places precision, not 64 or 128 bits :-)

The decimal32/64/128 types can store the number in this form. Unlike with binary floats the trailing zeros are not discarded.

The information about the intended precision is implicit in the literal. You see this done in exchange rates, stock prices, prices at your grocery store, etc.

The decimals can then be transformed into ones with higher precision during calculations and then possibly back to lower precision, but this is an explicit decision by the system doing the calculation.

The current behaviour will not be changed in this respect. There are two separate things that you are referring to though. One is the precision information associated with an individual Decimal e.g. the number of significant digits even if some are trailing zeros. The Decimal type preserves this information and so do the decimal64/128 formats:

Decimal('10.1000') Decimal('10.1000') Decimal('10.1000e-5') Decimal('0.000101000')

The other precision is the limiting precision of the current arithmetic context that is used when rounding. If you have a legal obligation to make that rounding be just so then you're into what I would consider to be expert usage of the decimal module.

Anyway, just throwing in some additional entropy into this discussion. Probably not all that helpful, since you're already converging on two possible solutions :-)

Thanks!

Oscar

12 Mar
12 Mar

1:11 p.m.

On 11.03.2014 14:04, Oscar Benjamin wrote:

On 11 March 2014 12:33, M.-A. Lemburg mal@egenix.com wrote: >

[...examples from finance...]

Anyway, just throwing in some additional entropy into this discussion. Probably not all that helpful, since you're already converging on two possible solutions :-)

Thanks!

Thanks for the detailed response. It's good to know that things will work out fine :-)

Marc-Andre Lemburg eGenix.com

Professional Python Services directly from the Source (#1, Mar 12 2014)

Python Projects, Consulting and Support ... http://www.egenix.com/ mxODBC.Zope/Plone.Database.Adapter ... http://zope.egenix.com/ mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/

2014-03-29: PythonCamp 2014, Cologne, Germany ... 17 days to go 2014-04-09: PyCon 2014, Montreal, Canada ... 28 days to go

::::: Try our mxODBC.Connect Python Database Interface for free ! ::::::

eGenix.com Software, Skills and Services GmbH Pastor-Loeh-Str.48 D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg Registered at Amtsgericht Duesseldorf: HRB 46611 http://www.egenix.com/company/contact/

11 Mar
11 Mar

9:44 p.m.

On 03/11/2014 07:33 AM, M.-A. Lemburg wrote:

On 11.03.2014 12:46, Nick Coghlan wrote:

...[Oscar's proposal being that picking just one decimal floating
point precision as*the* floating point precision]...

That approach buys us a couple of key benefits:

- they would be true constants, so constant folding, etc, would work normally, and you could tell from reading the code exactly what it will do, without needing to worry about the impact of thread local state
- you could generally use the builtin decimal literals without learning anything about decimal contexts, because decimal literals wouldn't be context dependent (although their behaviour would be formally defined in terms of an IEEE decimal context, for most users it would just be "this is how Python decimal literals behave")

The guidance to new users would then be*don't* use the decimal
module, use decimal literals instead. If you need more configurable
behaviour,*then* reach for the decimal module. However, the explicit
methods on decimal context objects should probably still be updated to
accept both fixed and variable precision decimals under this model.
I think you are leaving out one of the most important use cases
for decimal values: data input and output.

The literals would only appear in programs. However, in most use cases, you want to read decimal data from some source, process it and then write it back again. This would most likely also require using a few decimal literals, but the main source of decimals would still be constructors that you choose when writing the tools.

Coming back to floats and the decimal constructor:

In investment banking, for example, people usually work with floats all the time. You only convert back to decimals at the very end of some calculation. The reason here being either that the calculations involve complex operations which are not available for decimals, trying to keep error intervals small, or a combination of both. In general, you try to use as much precision as you can afford (in terms of memory and speed), to keep those error intervals small.

In accounting, you often use decimals for storing data with an (usually contractually or law based) agreed upon precision and rounding logic. However, there situations where you have to go to floats as well in order to run calculations, e.g. for interest, taxes, etc.

In both situations, you want to have the decimal constructor take the float values with full precision and only then apply the necessary rounding to turn the value into a form which complies with the agreed upon rules. It's also not uncommon to add correctional bookings to address rounding issues explicitly (e.g. when calculating VAT of a large number of individual items).

In short: you try to prevent rounding from happening as much as possible and when using it, you use it in a controlled way.

Based on this, the choice to have the decimal constructor use full precision when reading floats is a good one, even though it may not feel right for the novice, the casual decimal user or as human concept:-)

For decimal literals, I'd argue that if you enter a value 1.2500d, you are expecting a decimal with 4 decimal places precision, not 64 or 128 bits:-)

The information about the intended precision is implicit in the literal. You see this done in exchange rates, stock prices, prices at your grocery store, etc.

The decimals can then be transformed into ones with higher precision during calculations and then possibly back to lower precision, but this is an explicit decision by the system doing the calculation.

Anyway, just throwing in some additional entropy into this discussion. Probably not all that helpful, since you're already converging on two possible solutions:-)

I like django's measurement object approach.

https://docs.djangoproject.com/en/dev/ref/contrib/gis/measure/

It just needs a way to track accuracy (significant digits). Which can be independent of the underlying numeric type. (I think django has that too.)

Possibly, it could promote it's underlying type (int, float, decimal) if it needs to, or raise an exception if the underlying type isn't accurate enough to provide the requested precision.

The same concept could be used for currency calculations, or time for that matter.

This would do much more than adding a new underlying type to python. It encapsulates expert knowledge into objects that can be used by user who's expertise may be in other areas.

The actual underlying numeric type could depend on the amount of accuracy needed rather than some preconceived notion of greater accuracy that probably isn't needed in most cases. Ie.. Decimal128 for calculations that only require 6 significant digits.

Cheers, Ron

1:30 p.m.

Le 11/03/2014 11:51, Oscar Benjamin a écrit : >

When it comes to non-integer types there's just no
single good way of
doing it. So just in the stdlib we already have float, Fraction and
Decimal. The additional libraries that I use have many more numeric
types than that. The idea here is that for most users decimal128 just
is *the* decimal type.

Who are those "most users"? Those who are currently happy with float?

Defining two different but similar types, one for "beginner use" and one
for "expert use" is a terrible design pattern. People *will* have to
know about both types. Some libraries will choose to return the beginner
type, others will choose to return the expert type. Users will have to
learn the differences, will have to learn how conversions work, etc.

The decimal.Decimal type is IMO overly complex and should really be for niche use-cases like other non-stdlib multi-precision numeric types.

What is complex about it? The notion of a decimal's precision in number of digits is understandable by a 10 year old. It is arguably much simpler than unicode and character encodings. Besides, for most applications Decimal's current precision will be good enough: changing the current context (which, I believe, is thread-local, not process-global) is only necessary in special cases.

Java's BigDecimal is more user friendly in some important ways. It performs all calculations exactly until you explicitly round.

So if call BigDecimal(2).sqrt(), I get an infinite precision BigDecimal? :-)

(or perhaps it doesn't have .sqrt()?)

C++ is introducing decimal32, decimal64, and decimal128 as fixed-width decimal floating point types as according to IEEE-754-2008.

C++ has tons of integer types too, but Python has a single one.

Regards

Antoine.

1:42 p.m.

On 11 Mar 2014 23:31, "Antoine Pitrou" solipsis@pitrou.net wrote: >

Le 11/03/2014 11:51, Oscar Benjamin a écrit :

The decimal.Decimal type is IMO overly complex and should really be for niche use-cases like other non-stdlib multi-precision numeric types.

What is complex about it?

What is the result of "Decimal('1.0') + Decimal('1e70')"?

Correct answer: insufficient data (since we don't know the current precision).

That becomes a substantially more questionable answer for "1.0d + 1e70d". Having apparent numeric literals that can't be constant folded would be a much bigger design wart than having two different kinds of decimal.

Cheers, Nick.

2:40 p.m.

On 11 March 2014 13:30, Antoine Pitrou solipsis@pitrou.net wrote:

Le 11/03/2014 11:51, Oscar Benjamin a écrit : >

The decimal.Decimal type is IMO overly complex and should really be for niche use-cases like other non-stdlib multi-precision numeric types.

What is complex about it? The notion of a decimal's precision in number of digits is understandable by a 10 year old.

I agree, that's why I say that we just have a type with a fixed maximum number of digits. I think lots of people will be able to understand that.

It is arguably much simpler than unicode and character encodings. Besides, for most applications Decimal's current precision will be good enough: changing the current context (which, I believe, is thread-local, not process-global) is only necessary in special cases.

The problem is that the context cannot be depended upon so the result
of trivial calculations or even a negative decimal literal will depend
on something that can be changed *anywhere* else in the code. The
expression -1.123456789d will evaluate differently depending on the
context and I think many people will be surprised by that. I expect
that this would lead to naive code that breaks when the context is
changed. Great care needs to be taken to get this right, or otherwise
it should just be considered generally unsafe to change the context.

Java's BigDecimal is more user friendly in some important ways. It performs all calculations exactly until you explicitly round.

So if call BigDecimal(2).sqrt(), I get an infinite precision BigDecimal? :-)

(or perhaps it doesn't have .sqrt()?)

Apparently it does not: http://stackoverflow.com/questions/13649703/square-root-of-bigdecimal-in-jav...

But let's take something similar: BigDecimal(1).divide(BigDecimal(3)). This expression cannot be computed exactly in decimal format so it raises ArithmeticError. If you want it to round to some precision then you can provide a context with BigDecimal(1).divide(BigDecimal(3), mymathcontext). This rounds as specified in mymathcontext and the result of the expression is not affected by someone else changing a global arithmetic context.

Oscar

3:14 p.m.

It is arguably much simpler than unicode and character encodings. Besides, for most applications Decimal's current precision will be good enough: changing the current context (which, I believe, is thread-local, not process-global) is only necessary in special cases.

The problem is that the context cannot be depended upon so the result
of trivial calculations or even a negative decimal literal will depend
on something that can be changed *anywhere* else in the code. The
expression -1.123456789d will evaluate differently depending on the
context and I think many people will be surprised by that. I expect
that this would lead to naive code that breaks when the context is
changed. Great care needs to be taken to get this right, or otherwise
it should just be considered generally unsafe to change the context.

This is a good point, but I still don't think it outweighs the large cognitive baggage (and maintenance overhead for library authors as well as ourselves) of having two different types doing mostly similar things.

It is also not clear that a decimal type would be really beneficial for non-expert users. Most people get by with floats fine.

But let's take something similar: BigDecimal(1).divide(BigDecimal(3)). This expression cannot be computed exactly in decimal format so it raises ArithmeticError. If you want it to round to some precision then you can provide a context with BigDecimal(1).divide(BigDecimal(3), mymathcontext).

But then it becomes, ironically, more cumbersome to spell than Decimal(1) / Decimal(3) (I'm talking about the explicit context, of course, not the .divide() method which needn't exist in Python)

Regards

Antoine.