Re: [Python-Dev] [Python-ideas] Rough draft: Proposed format specifier for a thousands separator (discussion moved from python-dev)
Here's an update incorporating all the comments received so far. * Put into PEP format * Fixed typos * The suggestion for modifying the locale module was dropped. * The "n" specifier in the local module was referenced * Fixed minimumwidth --> width * PERIOD --> DOT * Added suggestions by Lie Ryan and Eric Smith ----------------------------------------------------------- PEP: XXX Title: Format Specifier for Thousands Separator Version: $Revision$ Last-Modified: $Date$ Author: Raymond Hettinger <python at rcn.com> Status: Draft Type: Standards Track Content-Type: text/x-rst Created: 12-Mar-2009 Post-History: 12-Mar-2009 Motivation ========== Provide a simple, non-locale aware way to format a number with a thousands separator. Adding thousands separators is one of the simplest ways to improve the professional appearance and readability of output exposed to end users. In the finance world, output with commas is the norm. Finance users and non-professional programmers find the locale approach to be frustrating, arcane and non-obvious. It is not the goal to replace locale or to accommodate every possible convention. The goal is to make a common task easier for many users. Current Version of the Mini-Language ==================================== * `Python 2.6 docs`_ .. _Python 2.6 docs: http://docs.python.org/library/string.html#formatstrings * PEP 3101 Advanced String Formatting Research so far =============== Scanning the web, I've found that thousands separators are usually one of COMMA, DOT, SPACE, or UNDERSCORE. When a COMMA is the decimal separator, the thousands separator is typically a DOT or SPACE (see examples from Denis Spir). James Knight observed that Indian/Pakistani numbering systems group by hundreds. Ben Finney noted that Chinese group by ten-thousands. Eric Smith pointed-out that these are already handled by the "n" specifier in the locale module (albiet only for integers). Visual Basic and its brethren (like MS Excel) use a completely different style and have ultra-flexible custom format specifiers like: "_($* #,##0_)". Proposal I (from Nick Coghlan) ============================== A comma will be added to the format() specifier mini-language: [[fill]align][sign][#][0][width][,][.precision][type] The ',' option indicates that commas should be included in the output as a thousands separator. As with locales which do not use a period as the decimal point, locales which use a different convention for digit separation will need to use the locale module to obtain appropriate formatting. The proposal works well with floats, ints, and decimals. It also allows easy substitution for other separators. For example:: format(n, "6,f").replace(",", "_") This technique is completely general but it is awkward in the one case where the commas and periods need to be swapped:: format(n, "6,f").replace(",", "X").replace(".", ",").replace("X", ".") Proposal II (to meet Antoine Pitrou's request) ============================================== Make both the thousands separator and decimal separator user specifiable but not locale aware. For simplicity, limit the choices to a comma, period, space, or underscore. [[fill]align][sign][#][0][width][T[tsep]][dsep precision][type] Examples:: format(1234, "8.1f") --> ' 1234.0' format(1234, "8,1f") --> ' 1234,0' format(1234, "8T.,1f") --> ' 1.234,0' format(1234, "8T .f") --> ' 1 234,0' format(1234, "8d") --> ' 1234' format(1234, "8T,d") --> ' 1,234' This proposal meets mosts needs (except for people wanting grouping for hundreds or ten-thousands), but it comes at the expense of being a little more complicated to learn and remember. Also, it makes it more challenging to write custom __format__ methods that follow the format specification mini-language. No change is proposed for the local module. Other Ideas =========== * Lie Ryan suggested a convenience function of the form:: create_format(self, type='i', base=16, seppos=4, sep=':', \ charset='0123456789abcdef', maxwidth=32, \ minwidth=32, pad='0') * Eric Smith would like the C version of the mini-language parser to be exposed. That would make it easier to write custom __format__ methods. Copyright ========= This document has been placed in the public domain. .. Local Variables: mode: indented-text indent-tabs-mode: nil sentence-end-double-space: t fill-column: 70 coding: utf-8 End:
Fixed typo in the example with spaces and commas. Discussion draft at: http://www.python.org/dev/peps/pep-0378/
Raymond Hettinger wrote:
Eric Smith pointed-out that these are already handled by the "n" specifier in the locale module (albiet only for integers).
It's supported by float, but it's just not very useful. For Decimal it's unsupported. Maybe this isn't a distinction worth pointing out.
Proposal I (from Nick Coghlan) ============================== ... [[fill]align][sign][#][0][width][,][.precision][type]
Proposal II (to meet Antoine Pitrou's request) ============================================== ... [[fill]align][sign][#][0][width][T[tsep]][dsep precision][type]
I was going to suggest that since the locale name for this is "grouping", we use "G". But since we're not doing a general-purpose grouping implementation, I think "T" better says "we're doing thousands, not general grouping. Perhaps this should go in a rationale section if we opt for "T". Now that I think about it, "G" is already a valid type, so it wouldn't work, anyway.
format(1234, "8T,d") --> ' 1,234'
For proposal 2, this case is unfortunate. Because for integers, there is no decimal allowed in the mini-language (it's currently illegal to use "8.1d"), you'd only ever add the thousands, but you'd always need the "T". It would be nice to come up with a specification that would degrade for integers such that "8,d" would give ' 1,234'. Proposal 1 is much nicer in that regard, although I definitely like the fact that the actual characters used for DOT and COMMA can be specified with proposal 2. Maybe you'd never really use "T,", since the comma is redundant, and you'd say: format(1234, "8Td") --> ' 1,234' in normal use. But "d" is also the default, so it just becomes: format(1234, "8T") --> ' 1,234' I like approach 2 in general. I'll give some thought to other, similar schemes which would allow "8," or "8,d" to work. I think people will write "8," and expect " 1,234", not an error. Eric.
Eric Smith wrote:
I like approach 2 in general. I'll give some thought to other, similar schemes which would allow "8," or "8,d" to work. I think people will write "8," and expect " 1,234", not an error.
Given that PEP 3101 deliberately exposed the underlying Formatter class to support more flexible expansions to formatting, I think trying to build that level of flexibility into the formatting mini-language itself would be a mistake. So step 1 would be to add the "," or "'" to the format string to indicate the use of separators. While both violate the rule of syntax not looking like grit on Tim's monitor, the latter has the virtues of not only being harder to confuse with a period, but also being consistent with the printf() grouping flag as defined in the Single Unix Specification v2 (as noted by JYK earlier in the thread and described in more detail at [1]). Then step 2 (if deemed appropriate) would be to expose the mini-language parser in a way that makes it easier to write a custom format_field() method in a Formatter subclass without having to completely duplicate the existing parser's functionality. Step 1 makes sense even if step 2 is never taken. Step 2 also makes sense even if step 1 is never taken. So I don't think the fact that step 2 is probably a good idea should be used to hold up PEP 378, nor do I think PEP 378 should be complicated by locale related issues that are better handled by making it easier to write a custom format_field() method in a Formatter subclass. Cheers, Nick. [1] http://linux.die.net/man/3/printf P.S. This post is really just an elaboration of this comment from JYK: """I'm not against Raymond's proposal, just against doing a *bad* job of making it work in multiple locales. Locale conventions can be complex, and are going to be best represented outside the format string.""" -- Nick Coghlan | ncoghlan@gmail.com | Brisbane, Australia ---------------------------------------------------------------
participants (3)
-
Eric Smith -
Nick Coghlan -
Raymond Hettinger