[Python-Dev] PEP 498 f-string: is it a preprocessor?

[Eric V. Smith]

On 08/10/2015 10:18 AM, Victor Stinner wrote:
> Hi,
> 
> I read the PEP but I don't understand how it is implemented. For me, it
> should be a simple preprocessor:
> 
> - f'x={x}' is replaced with 'x={0}'.format(x) by the compiler
> - f'x={1+1}' is replaced with 'x={0}'.format(1+1)
> - f'x={foo()!r}' is replaced with 'x={0!r}'.format(foo())
> - ...
> 
> That's all. No new language, no new function or method.

There is no new function or method being proposed. The "pre-processor"
is being implemented as the ast is being built. As the PEP says, though,
the expressions supported aren't exactly the same, so a simple
conversion to str.format syntax isn't possible.

> It's unclear to me if arbitrary expressions should be allowed or not. If
> not, we may pass parameters by keywords instead:
> 
> f'x={x}' is replaced with 'x={x}'.format(x=x) by the compiler
> 
> '...'.format(...) is highly optimized. In the current PEP, I see that
> each parameter is rendered in its own independent buffer (.__format__()
> method called multiple times) and then concateneted by ''.join(...).
> It's less efficient that using a single call to .format().

Well, that's what str.format() does: calls __format__ on each expression
and concatenates the results. Except str.format() uses _PyUncicodeWriter
rather than ''.join, so it skips creating the list.

I guess there is a short-circuit in format() where it has an exact
object check for string, float, double (and I think complex), in which
case it will skip the object allocation and basically call the internals
of the object's __format__ method. I could make a similar optimization
here, but it would require a new opcode. I'd like to look at some
benchmarks first.

I don't think such an optimization should drive acceptance or not: let's
decide on the functionality first.

Eric.