%-formatting with Decimals
During recent discussions I came across something unfortunate with formatting Decimals. Formatting with .format works as I would want:
from decimal import Decimal as D '{:1.0e}'.format(D('1.123e+1000')) '1e+1000' '{:1.0e}'.format(D('1.123e-1000')) '1e-1000' '{:.50f}'.format(D('1.1')) '1.10000000000000000000000000000000000000000000000000'
But %-formatting coerces to float:
'%1.0e' % D('1.123e-1000') '0e+00' '%1.0e' % D('1.123e+1000') 'inf' '%.50f' % D('1.1') '1.10000000000000008881784197001252323389053344726562'
Am I doing this wrong? Is this just a limitation of the old-style % formatting or something that could possibly be improved? Oscar
11.03.14 14:44, Oscar Benjamin написав(ла):
Am I doing this wrong? Is this just a limitation of the old-style % formatting or something that could possibly be improved?
Yes, this is a limitation of the old-style % formatting. Another example:
'{:.2f}'.format(1.234j) '0.00+1.23j' '%.2f' % 1.234j Traceback (most recent call last): File "<stdin>", line 1, in <module> TypeError: can't convert complex to float
On Tue, Mar 11, 2014, at 9:15, Serhiy Storchaka wrote:
11.03.14 14:44, Oscar Benjamin написав(ла):
Am I doing this wrong? Is this just a limitation of the old-style % formatting or something that could possibly be improved?
Yes, this is a limitation of the old-style % formatting. Another example:
Is % formatting not going to be updated to support new features or fix bugs like this? Is it going to be deprecated? Will a tool be provided to convert usages (at least with static format strings) to str.format()?
On Mar 11, 2014, at 12:17, random832@fastmail.us wrote:
On Tue, Mar 11, 2014, at 9:15, Serhiy Storchaka wrote:
11.03.14 14:44, Oscar Benjamin написав(ла):
Am I doing this wrong? Is this just a limitation of the old-style % formatting or something that could possibly be improved?
Yes, this is a limitation of the old-style % formatting. Another example:
Is % formatting not going to be updated to support new features or fix bugs like this?
That's an inherent limitation of % formatting: it uses printf-style type codes, whose meaning is baked into the % operator, rather than calling __format__ on its arguments to handle it polymorphically. %f means treat the argument as a float. That limited flexibility is part of what allows it to be simpler, faster, able to share some format strings with C, etc. It's doing what it was designed to do; the fact that it wasn't designed the same as str.format isn't a bug. That's not to say that it couldn't be extended with new type codes for decimals. If C1x adds printf type codes (or if gcc or some other compiler creates its own extension), you could make a pretty good argument for % formatting to borrow them.
Is it going to be deprecated?
It wasn't deprecated in 3.0, and after the issue was raised again, I'm pretty sure the decision was that deprecation is off the table until at least Python 4000. And if you go back and read the arguments, the reason it wasn't deprecated is that for some use cases it's better, in part because it's not extensible, and has the corresponding advantages mentioned above.
Will a tool be provided to convert usages (at least with static format strings) to str.format()?
I'm pretty sure people have already written such tools, back when str.format was first created. I don't think too many people use them. If %-formatting is working for your existing code, or appropriate for some new code, there's no reason to change it. If {}-formatting is appropriate for some new code, it's easier to write it directly than to try to write a different format and then convert it.
On 03/11/2014 12:17 PM, random832@fastmail.us wrote:
On Tue, Mar 11, 2014, at 9:15, Serhiy Storchaka wrote:
11.03.14 14:44, Oscar Benjamin написав(ла):
Am I doing this wrong? Is this just a limitation of the old-style % formatting or something that could possibly be improved?
Yes, this is a limitation of the old-style % formatting. Another example:
Is % formatting not going to be updated to support new features
No.
or fix bugs like this?
It's not a bug. Decimal is not float.
Is it going to be deprecated?
No. %-formatting has its place.
Will a tool be provided to convert usages (at least with static format strings) to str.format()?
No. Feel free to write one and share it with the community. :) Basically, %-formatting is primitive, .format-formatting is advanced. If you want advanced behavior, use the advanced method. -- ~Ethan~
On Tue, Mar 11, 2014, at 16:25, Ethan Furman wrote:
Is % formatting not going to be updated to support new features
No.
or fix bugs like this?
It's not a bug. Decimal is not float.
The bug to which I was referring involves not calling __format__ on whatever object is passed in as str.format does - Decimal not being float is no impediment to str.format and Decimal.__format__ accepting the same format strings that float.__format__ does.
Is it going to be deprecated?
No. %-formatting has its place.
What place is that? There should be one-- and preferably only one --obvious way to do it. We've got two syntaxes, and two functions, that do _almost_ the same thing, but with two completely different microlanguages. Why?
On 03/11/2014 02:51 PM, random832@fastmail.us wrote:
On Tue, Mar 11, 2014, at 16:25, Ethan Furman wrote:
at some point prior, random832@fastmail.us wrote:
Is % formatting not going to be updated to support new features
No.
or fix bugs like this?
It's not a bug. Decimal is not float.
The bug to which I was referring involves not calling __format__ on whatever object is passed in as str.format does - Decimal not being float is no impediment to str.format and Decimal.__format__ accepting the same format strings that float.__format__ does.
That is not a bug. str.__mod__ is not documented as calling an object's __format__ method, and indeed it does not.
Is it going to be deprecated?
No. %-formatting has its place.
What place is that?
There should be one-- and preferably only one --obvious way to do it.
You'll notice the word 'preferably' in there. The preferred method is format. %-formatting has a good place in lazy evaluation (in logging, for example), as well as other areas.
We've got two syntaxes, and two functions, that do _almost_ the same thing, but with two completely different microlanguages. Why?
Because % was there first, but had some failings that are addressed by format. -- ~Ethan~
On 11 March 2014 21:58, Ethan Furman <ethan@stoneleaf.us> wrote:
On 03/11/2014 02:51 PM, random832@fastmail.us wrote:
The bug to which I was referring involves not calling __format__ on whatever object is passed in as str.format does - Decimal not being float is no impediment to str.format and Decimal.__format__ accepting the same format strings that float.__format__ does.
That is not a bug. str.__mod__ is not documented as calling an object's __format__ method, and indeed it does not.
Instead it calls float on the object:
class A: ... def __float__(self): ... print('__float__ called') ... '%f' % A() __float__ called Traceback (most recent call last): File "<stdin>", line 1, in <module> TypeError: nb_float should return float object
There's no mention of that anywhere in the docs though: http://docs.python.org/3.4/library/stdtypes.html#printf-style-string-formatt... There's also no mention of this in the decimal docs. Oscar
On 03/11/2014 03:49 PM, Oscar Benjamin wrote:
On 11 March 2014 21:58, Ethan Furman <ethan@stoneleaf.us> wrote:
On 03/11/2014 02:51 PM, random832@fastmail.us wrote:
The bug to which I was referring involves not calling __format__ on whatever object is passed in as str.format does - Decimal not being float is no impediment to str.format and Decimal.__format__ accepting the same format strings that float.__format__ does.
That is not a bug. str.__mod__ is not documented as calling an object's __format__ method, and indeed it does not.
Instead it calls float on the object:
class A: ... def __float__(self): ... print('__float__ called') ... '%f' % A() __float__ called
Well, you did ask for a float, and how else are you going to get one? ;)
Traceback (most recent call last): File "<stdin>", line 1, in <module> TypeError: nb_float should return float object
There's no mention of that anywhere in the docs though: http://docs.python.org/3.4/library/stdtypes.html#printf-style-string-formatt...
We should probably fix that. -- ~Ethan~
On Mar 11, 2014, at 15:49, Oscar Benjamin <oscar.j.benjamin@gmail.com> wrote:
On 11 March 2014 21:58, Ethan Furman <ethan@stoneleaf.us> wrote:
On 03/11/2014 02:51 PM, random832@fastmail.us wrote:
The bug to which I was referring involves not calling __format__ on whatever object is passed in as str.format does - Decimal not being float is no impediment to str.format and Decimal.__format__ accepting the same format strings that float.__format__ does.
That is not a bug. str.__mod__ is not documented as calling an object's __format__ method, and indeed it does not.
Instead it calls float on the object:
class A: ... def __float__(self): ... print('__float__ called') ... '%f' % A() __float__ called Traceback (most recent call last): File "<stdin>", line 1, in <module> TypeError: nb_float should return float object
There's no mention of that anywhere in the docs though: http://docs.python.org/3.4/library/stdtypes.html#printf-style-string-formatt...
I wouldn't expect it to mention that it calls the __float__ method (much less the nb_float C slot), because it doesn't. It converts to float in the normal way, by calling the float constructor or it's C equivalent. And the docs for float say pretty clearly that "For a general Python object" (that is, not a string or a float), "float(x) delegates to x.__float__()". Maybe section 4.7.2 could be more explicit about the fact that the conversion type specifies a conversion to the appropriate type (int, float, or str)?
There's also no mention of this in the decimal docs.
Why should there be? Neither printf-style string formatting nor the float constructor have anything to do with decimals.
On 12 March 2014 00:45, Andrew Barnert <abarnert@yahoo.com> wrote:
On Mar 11, 2014, at 15:49, Oscar Benjamin <oscar.j.benjamin@gmail.com> wrote:
On 11 March 2014 21:58, Ethan Furman <ethan@stoneleaf.us> wrote:
On 03/11/2014 02:51 PM, random832@fastmail.us wrote:
The bug to which I was referring involves not calling __format__ on whatever object is passed in as str.format does - Decimal not being float is no impediment to str.format and Decimal.__format__ accepting the same format strings that float.__format__ does.
That is not a bug. str.__mod__ is not documented as calling an object's __format__ method, and indeed it does not.
Instead it calls float on the object:
class A: ... def __float__(self): ... print('__float__ called') ... '%f' % A() __float__ called Traceback (most recent call last): File "<stdin>", line 1, in <module> TypeError: nb_float should return float object
There's no mention of that anywhere in the docs though: http://docs.python.org/3.4/library/stdtypes.html#printf-style-string-formatt...
I wouldn't expect it to mention that it calls the __float__ method (much less the nb_float C slot), because it doesn't.
It converts to float in the normal way, by calling the float constructor or it's C equivalent. And the docs for float say pretty clearly that "For a general Python object" (that is, not a string or a float), "float(x) delegates to x.__float__()".
Maybe section 4.7.2 could be more explicit about the fact that the conversion type specifies a conversion to the appropriate type (int, float, or str)?
Yes that's what I meant. The exact words used there to describe the 'f' format code are 'Floating point decimal format'. There's no mention that it will first need to coerce to a binary float before converting that to decimal. (Perhaps those words predate the decimal module).
There's also no mention of this in the decimal docs.
Why should there be? Neither printf-style string formatting nor the float constructor have anything to do with decimals.
Because it's a relevant issue to consider when working with Decimals. In simple cases it will work but in other cases it won't. Oscar
On 03/12/2014 06:13 AM, Oscar Benjamin wrote:
On 12 March 2014 00:45, Andrew Barnert <abarnert@yahoo.com> wrote:
On Mar 11, 2014, at 15:49, Oscar Benjamin <oscar.j.benjamin@gmail.com> wrote:
There's also no mention of this in the decimal docs.
Why should there be? Neither printf-style string formatting nor the float constructor have anything to do with decimals.
Because it's a relevant issue to consider when working with Decimals. In simple cases it will work but in other cases it won't.
In the hopefully unlikely event we do get decimal literals, I think it would be reasonable to extend %f-formatting (and its friends e and g) to cover decimals as well as floats. Eric.
On 12 Mar 2014 07:51, <random832@fastmail.us> wrote:
On Tue, Mar 11, 2014, at 16:25, Ethan Furman wrote:
Is % formatting not going to be updated to support new features
No.
or fix bugs like this?
It's not a bug. Decimal is not float.
The bug to which I was referring involves not calling __format__ on whatever object is passed in as str.format does - Decimal not being float is no impediment to str.format and Decimal.__format__ accepting the same format strings that float.__format__ does.
Is it going to be deprecated?
No. %-formatting has its place.
What place is that?
There should be one-- and preferably only one --obvious way to do it.
We've got two syntaxes, and two functions, that do _almost_ the same thing, but with two completely different microlanguages. Why?
Start by reading PEP 3101, which added str.format to address the limitations of mod-formatting. For new users, the recommendation is unconditional: use str.format. mod-formatting is now a power tool for optimisation, consistency with C implementations, templating format strings without excessive escaping and (the main reason it still exists), backwards compatibility (both for existing code and existing programmers). The original plan was to eventually deprecate mod-formatting entirely, but not only did we discover that was harder than expected, we also found it was actually incredibly useful in some cases to have a formatting system available that relied on coercion rather than polymorphism, since it could be faster, was more predictable and also more easily mapped to other languages like C (especially since Python's mod-formatting is based on C's printf syntax). That's why the binary interpolation proposal for 3.5 now *only* proposes restoring mod-formatting for the binary domain, leaving str.format as a text domain only operation. Cheers, Nick.
_______________________________________________ Python-ideas mailing list Python-ideas@python.org https://mail.python.org/mailman/listinfo/python-ideas Code of Conduct: http://python.org/psf/codeofconduct/
On Wed, Mar 12, 2014 at 9:30 AM, Nick Coghlan <ncoghlan@gmail.com> wrote:
mod-formatting is now a power tool for optimisation, consistency with C implementations, templating format strings without excessive escaping and (the main reason it still exists), backwards compatibility (both for existing code and existing programmers).
And consistency with other C-derived languages, too. The only weirdness of Python's that isn't found in others is the "hey look isn't it cool" use of the % operator, which results in edge cases because the operator always takes exactly one argument on the RHS - which can be solved by turning it into a function:
def sprintf(fmt, *args): return fmt%args
sprintf("foo %s bar",(1,2,3)) 'foo (1, 2, 3) bar' sprintf("foo %d bar %d quux",1,2) 'foo 1 bar 2 quux'
ChrisA
participants (8)
-
Andrew Barnert -
Chris Angelico -
Eric V. Smith -
Ethan Furman -
Nick Coghlan -
Oscar Benjamin -
random832＠fastmail.us -
Serhiy Storchaka