Comprehensions within f-strings
Recently there's been some discussion around string comprehensions, and I wanted to look at a specific variant of the proposal in a bit more detail. Original thread: https://mail.python.org/archives/list/python-ideas@python.org/thread/MVQGP4G... Let's say i have a matrix of numbers: matrix = [[randint(7, 50) / randint(1, 3) for _ in range(4)] for _ in range(4)] I want to format and display each row so that the columns are nicely lined up. Maybe also display the sum of the row at the end of each line: for row in matrix: print(''.join(f'{n:>8.3f}' for n in row) + f' | {sum(row):>8.3f}') This gives me a nicely formatted table. Now with the proposal: for row in matrix: print(f'{n for n in row:>8.3f} | {sum(row):>8.3f}') The idea is that you would be able to embed a comprehension in f-string interpolations, and that the format specifier at the end would be applied to all the generated items. This has a few advantages compared to the first version. It's a bit shorter and I find it easier to see what kind of shape the output will look like. It would also be faster since the interpreter would be able to append the formatted numbers directly to the final string. The first version needs to create a temporary string object for each number and then feed it into the iterator protocol before joining them together.
On Sun, May 02, 2021 at 04:09:21AM -0000, Valentin Berlier wrote:
As a general rule, we should avoid needless generator expressions that just iterate over themselves: n for n in row is just the same as row except it creates a pointless generator to iterate over something that is already iterable. Your proposed f-string syntax: f'{n for n in row:>8.3f} | {sum(row):>8.3f}' is already legal except for the first format specifier. Removing that: f'{n for n in row} | {sum(row):>8.3f}' gives us working code. I don't believe that we ought to confuse the format specifier syntax by making the f format code have magical powers when given an in-place generator expression, and otherwise have the regular meaning for anything else. Better to invent a new format code, which I'm going to spell as 'X' for lack of something better, that maps a format specifier to every element of any iterable (not just generator comprehensions): f'{row:X>8.3f} | {sum(row):>8.3f}' meaning, format each element of row with `>8.3f`.
The idea is that you would be able to embed a comprehension in f-string interpolations,
When the only tool you have is a hammer, every problem looks like it is crying out for a comprehension *wink* -- Steve
On 5/2/2021 5:44 AM, Steven D'Aprano wrote:
I'm completely opposed to all of this, but I'll note that you don't want the thing that gives instructions to the f-string mechanism to be part of the format specifier, you'd want it to be a conversion flag like !r and !s. The format spec is reserved for the object being formatted. The way this is specified, you'd either have to change the f-string mechanics to start interpreting format specs, or have every iterable understand what "X" means. For example, "X" is already understood by datetime as part of its format spec "language":
f'{datetime.now() - timedelta(days=3):X days ago it was a %A}' 'X days ago it was a Thursday'
I don't think you'd want f-strings to hijack that expression because it starts with "X". Better to do something like: f'{row!X:>8.3f} | {sum(row):>8.3f}' But I'll reiterate that I'm opposed. Sometimes you just need to do the formatting outside of an f-string. Terseness shouldn't be the ultimate goal.
Or crying out for an f-string! The thing I'd like everyone to remember is that the format spec language as implemented by some (but not all) of the built-in types isn't the only format spec language around, and indeed it's not the only used by built-in types. -- Eric V. Smith
On Sun, May 02, 2021 at 04:30:38PM -0400, Eric V. Smith wrote: [I suggested]
Good point! Thank you.
That makes more sense.
But I'll reiterate that I'm opposed. Sometimes you just need to do the formatting outside of an f-string. Terseness shouldn't be the ultimate goal.
I agree with your general observation, but in the specific case of "do this to every element of this iterable", I think that's common enough that we should consider building it into the mini-languages used by format and f-strings. Python is an extremely readable language, and it is possible to be too terse as well as too verbose. Not everything needs to be a one-liner single expression. But if you see boilerplate code like: for each element of spam: format element in the exact same way it makes sense to consider abstracting it into a single operation: format each element of spam -- Steve
On Mon, May 3, 2021 at 9:41 AM Steven D'Aprano <steve@pearwood.info> wrote:
Agreed. Python is quite good at bulk operations, and this is something that definitely has value. I suspect it wouldn't fit into format(), but it would be perfect in f-strings. There are other string-formatting tools that allow a "do this for every element" embed. ChrisA
Eric V. Smith writes:
I wondered, is that even possible? OK, I guess you could do it with a Row(Any) class something like class Row: def __init__(self, row): self.row = row def __format__(self, spec): """ spec looks like "\t/\n/>8.3f" """ sep, end, spec = spec.split('/') # random bikeshed, # paint later return sep.join(format(self.row, spec)) + end but why bother defining !X to convert to Row when f'{Row(row)://>8.3f} | {sum(row):>8.3f}' should DTRT with the above class? EIBTI here, I think. (OTOH, with Cameron's idea of a programmable mapping of conversions, you could give Row a better name like "IterableFormatter", get the benefit of brevity with a conversion code, and still be explicit enough for me. YMMV)
"Terse enough" looks possible to me, though. Even without a new conversion. Steve
On Mon, May 3, 2021 at 5:37 AM Stephen J. Turnbull < turnbull.stephen.fw@u.tsukuba.ac.jp> wrote:
f'{row!X:>8.3f} | {sum(row):>8.3f}'
hmm -- this is making me think that maybe we should re-enable sum() for strings -- with an optimized path, rather than an TypeError. I understand the lesson that is supposed to be provided by making that an error, but I'm not sure it really helps anyone really learn anything. would that be more "intuitive" than "".join() maybe -- the fact that the Error was explicitly added implies that people were trying to do it. -CHB -- Christopher Barker, PhD (Chris) Python Language Consulting - Teaching - Scientific Software Development - Desktop GUI and Web Development - wxPython, numpy, scipy, Cython
I didn't even realize f'{n for n in row}' was already valid syntax. Since generator expressions can usually only appear within parentheses, I assumed the syntax wouldn't conflict with anything because you would need to parenthesize the generator to make it work. Anyway, now I see that the better option is to leave the interpolation as-is but introduce a conversion flag that can unroll any iterable, so the current behavior actually works in our favor here. This would indeed be more powerful than any specialized comprehension syntax. print(''.join(f'{n:>8.3f}' for n in row) + f' | {sum(row):>8.3f}') print(f'{row!u:>8.3f} | {sum(row):>8.3f}') Now the second option is even more attractive in my opinion. I'm using !u for "unroll". The good news is that implementing a conversion flag is a much less intrusive change.
On Sun, May 02, 2021 at 04:09:21AM -0000, Valentin Berlier wrote:
As a general rule, we should avoid needless generator expressions that just iterate over themselves: n for n in row is just the same as row except it creates a pointless generator to iterate over something that is already iterable. Your proposed f-string syntax: f'{n for n in row:>8.3f} | {sum(row):>8.3f}' is already legal except for the first format specifier. Removing that: f'{n for n in row} | {sum(row):>8.3f}' gives us working code. I don't believe that we ought to confuse the format specifier syntax by making the f format code have magical powers when given an in-place generator expression, and otherwise have the regular meaning for anything else. Better to invent a new format code, which I'm going to spell as 'X' for lack of something better, that maps a format specifier to every element of any iterable (not just generator comprehensions): f'{row:X>8.3f} | {sum(row):>8.3f}' meaning, format each element of row with `>8.3f`.
The idea is that you would be able to embed a comprehension in f-string interpolations,
When the only tool you have is a hammer, every problem looks like it is crying out for a comprehension *wink* -- Steve
On 5/2/2021 5:44 AM, Steven D'Aprano wrote:
I'm completely opposed to all of this, but I'll note that you don't want the thing that gives instructions to the f-string mechanism to be part of the format specifier, you'd want it to be a conversion flag like !r and !s. The format spec is reserved for the object being formatted. The way this is specified, you'd either have to change the f-string mechanics to start interpreting format specs, or have every iterable understand what "X" means. For example, "X" is already understood by datetime as part of its format spec "language":
f'{datetime.now() - timedelta(days=3):X days ago it was a %A}' 'X days ago it was a Thursday'
I don't think you'd want f-strings to hijack that expression because it starts with "X". Better to do something like: f'{row!X:>8.3f} | {sum(row):>8.3f}' But I'll reiterate that I'm opposed. Sometimes you just need to do the formatting outside of an f-string. Terseness shouldn't be the ultimate goal.
Or crying out for an f-string! The thing I'd like everyone to remember is that the format spec language as implemented by some (but not all) of the built-in types isn't the only format spec language around, and indeed it's not the only used by built-in types. -- Eric V. Smith
On Sun, May 02, 2021 at 04:30:38PM -0400, Eric V. Smith wrote: [I suggested]
Good point! Thank you.
That makes more sense.
But I'll reiterate that I'm opposed. Sometimes you just need to do the formatting outside of an f-string. Terseness shouldn't be the ultimate goal.
I agree with your general observation, but in the specific case of "do this to every element of this iterable", I think that's common enough that we should consider building it into the mini-languages used by format and f-strings. Python is an extremely readable language, and it is possible to be too terse as well as too verbose. Not everything needs to be a one-liner single expression. But if you see boilerplate code like: for each element of spam: format element in the exact same way it makes sense to consider abstracting it into a single operation: format each element of spam -- Steve
On Mon, May 3, 2021 at 9:41 AM Steven D'Aprano <steve@pearwood.info> wrote:
Agreed. Python is quite good at bulk operations, and this is something that definitely has value. I suspect it wouldn't fit into format(), but it would be perfect in f-strings. There are other string-formatting tools that allow a "do this for every element" embed. ChrisA
Eric V. Smith writes:
I wondered, is that even possible? OK, I guess you could do it with a Row(Any) class something like class Row: def __init__(self, row): self.row = row def __format__(self, spec): """ spec looks like "\t/\n/>8.3f" """ sep, end, spec = spec.split('/') # random bikeshed, # paint later return sep.join(format(self.row, spec)) + end but why bother defining !X to convert to Row when f'{Row(row)://>8.3f} | {sum(row):>8.3f}' should DTRT with the above class? EIBTI here, I think. (OTOH, with Cameron's idea of a programmable mapping of conversions, you could give Row a better name like "IterableFormatter", get the benefit of brevity with a conversion code, and still be explicit enough for me. YMMV)
"Terse enough" looks possible to me, though. Even without a new conversion. Steve
On Mon, May 3, 2021 at 5:37 AM Stephen J. Turnbull < turnbull.stephen.fw@u.tsukuba.ac.jp> wrote:
f'{row!X:>8.3f} | {sum(row):>8.3f}'
hmm -- this is making me think that maybe we should re-enable sum() for strings -- with an optimized path, rather than an TypeError. I understand the lesson that is supposed to be provided by making that an error, but I'm not sure it really helps anyone really learn anything. would that be more "intuitive" than "".join() maybe -- the fact that the Error was explicitly added implies that people were trying to do it. -CHB -- Christopher Barker, PhD (Chris) Python Language Consulting - Teaching - Scientific Software Development - Desktop GUI and Web Development - wxPython, numpy, scipy, Cython
I didn't even realize f'{n for n in row}' was already valid syntax. Since generator expressions can usually only appear within parentheses, I assumed the syntax wouldn't conflict with anything because you would need to parenthesize the generator to make it work. Anyway, now I see that the better option is to leave the interpolation as-is but introduce a conversion flag that can unroll any iterable, so the current behavior actually works in our favor here. This would indeed be more powerful than any specialized comprehension syntax. print(''.join(f'{n:>8.3f}' for n in row) + f' | {sum(row):>8.3f}') print(f'{row!u:>8.3f} | {sum(row):>8.3f}') Now the second option is even more attractive in my opinion. I'm using !u for "unroll". The good news is that implementing a conversion flag is a much less intrusive change.
participants (6)
-
Chris Angelico -
Christopher Barker -
Eric V. Smith -
Stephen J. Turnbull -
Steven D'Aprano -
Valentin Berlier