[Python-ideas] String Format Callable Flag (Was: Efficient Debug Logging)

[MRAB]

On 2017-02-17 16:37, Mark E. Haase wrote:
> In the debug logging thread, somebody suggested the "%r" format
> specifier and a custom __repr__. This is a neat solution because Python
> logging already includes a "delayed" evaluation of sorts: it formats the
> logging string *after* it determines that the message's log level is
> greater than or equal to the logger's level. However, wrapping up every
> expensive debugging computation in a new class might get tedious, so
> here's a strawman proposal for something lighter weight.
>
> Python has two string formatting mini-languages. Both allow formatting
> flags, for example in "%03d", the "0" (zero) is a flag that means to pad
> with leading zeroes. I propose to add a string format flag to both
> mini-languages that explicitly says, "this argument is callable, and its
> *return value* should be formatted."
>
> In current Python:
>
>     >>> expensive = lambda: 2
>     >>> logging.debug('%03d %03d', 1, expensive())
>
> In this case, the "expensive" code is executed even though nothing is
> logged.
>
>     >>> expensive = lambda: 2
>     >>> logging.root.setLevel(logging.DEBUG)
>     >>> logging.debug('%03d %03d', 1, expensive())
>     DEBUG:root:001 002
>
> With a different log level, the expensive code is executed and the
> message is logged.
>
> The suggested change is to add a "C" flag to the formatting language
> that indicates an argument is callable. When formatting the string, the
> argument will be called and its result will be used for the formatting
> operation. The C flag must be handled before any other flags, regardless
> of the order in which they are specified. The callable's return value is
> then formatted according to the rest of the specifier.
>
A letter usually marks the end of the format, e.g. '%03d', so '%C03d' 
could be read as a format '%C' (currently undefined) followed by '03d'. 
I'd suggest using some punctuation character instead.

It would also be a good idea for the 'new-style' format strings to use 
the same character.

> With this change, the above example can now be written as:
>
>     >>> expensive = lambda: 2
>     >>> logging.debug('%03d %C03d', 1, expensive)
>
> The "expensive" code is not executed at this log level.
>
>     >>> expensive = lambda: 2
>     >>> logging.root.setLevel(logging.DEBUG)
>     >>> logging.warn('%03d %C03d', 1, expensive)
>     WARNING:root:001 002
>
> The expensive code is *only* executed when the message will be logged.
> The callable's return value is formatted with the specifier '%03d', i.e.
> the same specifier as the callable but without the "C" flag.
>
>
>
> Pros:
>
> 1) Much smaller change to the language.
> 2) Simplifies a common (?) problem with logging.
> 3) Applies to all string formatting, not just logging. (But I'm not sure
> what the other use cases are.)
>
> Cons:
>
> 1) Doesn't generalize as well as the various "delayed" proposals. (But
> I'm not sure what other use cases "delayed" has.)
>