[Python-ideas] String Format Callable Flag (Was: Efficient Debug Logging)

[Mark E. Haase]

In the debug logging thread, somebody suggested the "%r" format specifier
and a custom __repr__. This is a neat solution because Python logging
already includes a "delayed" evaluation of sorts: it formats the logging
string *after* it determines that the message's log level is greater than
or equal to the logger's level. However, wrapping up every expensive
debugging computation in a new class might get tedious, so here's a
strawman proposal for something lighter weight.

Python has two string formatting mini-languages. Both allow formatting
flags, for example in "%03d", the "0" (zero) is a flag that means to pad
with leading zeroes. I propose to add a string format flag to both
mini-languages that explicitly says, "this argument is callable, and its
*return value* should be formatted."

In current Python:

    >>> expensive = lambda: 2
    >>> logging.debug('%03d %03d', 1, expensive())

In this case, the "expensive" code is executed even though nothing is
logged.

    >>> expensive = lambda: 2
    >>> logging.root.setLevel(logging.DEBUG)
    >>> logging.debug('%03d %03d', 1, expensive())
    DEBUG:root:001 002

With a different log level, the expensive code is executed and the message
is logged.

The suggested change is to add a "C" flag to the formatting language that
indicates an argument is callable. When formatting the string, the argument
will be called and its result will be used for the formatting operation.
The C flag must be handled before any other flags, regardless of the order
in which they are specified. The callable's return value is then formatted
according to the rest of the specifier.

With this change, the above example can now be written as:

    >>> expensive = lambda: 2
    >>> logging.debug('%03d %C03d', 1, expensive)

The "expensive" code is not executed at this log level.

    >>> expensive = lambda: 2
    >>> logging.root.setLevel(logging.DEBUG)
    >>> logging.warn('%03d %C03d', 1, expensive)
    WARNING:root:001 002

The expensive code is *only* executed when the message will be logged. The
callable's return value is formatted with the specifier '%03d', i.e. the
same specifier as the callable but without the "C" flag.



Pros:

1) Much smaller change to the language.
2) Simplifies a common (?) problem with logging.
3) Applies to all string formatting, not just logging. (But I'm not sure
what the other use cases are.)

Cons:

1) Doesn't generalize as well as the various "delayed" proposals. (But I'm
not sure what other use cases "delayed" has.)
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20170217/8a5d6e16/attachment-0001.html>