Change to logging Formatters: support for alternative format styles
I've just checked in a change to logging into the py3k branch (r85835), including doc changes and tests, for providing slightly more flexibility in alternative format styles for logging. Basically, Formatter.__init__ gets an extra optional keyword arg style=<one of '%' (default), '{' or '$'>. This is then used to merge the format string with the LogRecord: either fmt % record.__dict__, or fmt.format(**record.dict), or string.Template(fmt).substitute(**record.dict). Backward compatibility is maintained (unless I've missed something). This does not cater for how you combine the logging message + its args when you make a logging call, but should work with any existing library which uses %-formatting in its logging calls. We discussed this here around a year ago in the context of generally encouraging a move from %-formatting to {}-formatting, but there seemed to be no obvious easy path forward. As of now, and even without this change I've just checked in, developers can use {}- or $-formatting in their logging messages using e.g. logger.debug(__('Message with {} {}', 2, 'place-holders')) with a suitable class standing in for __. A bit ugly, I know, with extra parentheses and what not, but hopefully for those who must have {} now it shouldn't be too much of a price to pay and there's no major performance implication, as actual rendering to string is done as late as possible, just as it is now. Comments welcome. Assuming there are no strong objections asking for reversion of this change, I'll publicise to the wider community in a few days. Regards, Vinay Sajip
On Tue, Oct 26, 2010 at 12:28 AM, Vinay Sajip <vinay_sajip@yahoo.co.uk> wrote:
Comments welcome. Assuming there are no strong objections asking for reversion of this change, I'll publicise to the wider community in a few days.
It strikes me as a solid, pragmatic solution to a thorny problem. Looking at your checkin though, I wonder if it might be worth implementing some little formatting style classes to get rid of the if/elif chains from the Formatter code. Something like: # Style objects class PercentStyle(object): default_format = "%(message)" def is_asctime_in_format(fmt): return fmt.find("%(asctime)") >= 0 def format_record(fmt, record): return fmt % record.__dict__ class StrFormatStyle(object): default_format = "{message}" def is_asctime_in_format(fmt): return fmt.find("{asctime}") >= 0 def format_record(fmt, record): return fmt.format(**record.__dict__) from string import Template # Top level, embedding may cause import deadlock issues class StringTemplateStyle(object): def __init__(self): self._fmt = self._tmpl = None # Setup for format_record default_format = "${message}" def is_asctime_in_format(fmt): return fmt.find("$asctime") >= 0 or fmt.find("${asctime}") >= 0 def format_record(fmt, record): if fmt != self._fmt: self._fmt = fmt self._tmpl = Template(fmt) return self._tmpl.substitute(**record.__dict__) # Build style map _STYLE_CODES = tuple("% { $".split()) _STYLES = PercentStyle, StrFormatStyle, StringTemplateStyle _STYLE_MAP = dict(zip(_STYLE_CODES, _STYLES)) # Interpretation of the style parameter in Formatter.__init__ if style not in _STYLE_CODES: # Preserve option to allow arbitrary style objects at some point in the future raise ValueError("Style must be one of {!r}".format(_STYLE_CODES)) self._style = _STYLE_MAP[style]() # self._fmt initialisation if/elif chain replacement self._fmt = self._style.default_format # Time check if/elif chain replacement result = self._style.is_asctime_in_format(self._fmt) # Record formatting if/elif chain replacement s = self._style.format_record(self._fmt, record) Cheers, Nick. -- Nick Coghlan | ncoghlan@gmail.com | Brisbane, Australia
On Tue, Oct 26, 2010 at 9:08 PM, Nick Coghlan <ncoghlan@gmail.com> wrote:
On Tue, Oct 26, 2010 at 12:28 AM, Vinay Sajip <vinay_sajip@yahoo.co.uk> wrote:
Comments welcome. Assuming there are no strong objections asking for reversion of this change, I'll publicise to the wider community in a few days.
It strikes me as a solid, pragmatic solution to a thorny problem.
Looking at your checkin though, I wonder if it might be worth implementing some little formatting style classes to get rid of the if/elif chains from the Formatter code. Something like:
Some syntax highlighting may make that wall-o'-code a little easier to read: http://pastebin.com/SG0Qr3r5 Cheers, Nick. -- Nick Coghlan | ncoghlan@gmail.com | Brisbane, Australia
Line 31 (in Pastebin): _STYLE_CODES = tuple("% { $".split()) Is this really necessary? Why not _STYLE_CODES = ('%', '{', '$') On Tue, Oct 26, 2010 at 1:15 PM, Nick Coghlan <ncoghlan@gmail.com> wrote:
On Tue, Oct 26, 2010 at 9:08 PM, Nick Coghlan <ncoghlan@gmail.com> wrote:
On Tue, Oct 26, 2010 at 12:28 AM, Vinay Sajip <vinay_sajip@yahoo.co.uk> wrote:
Comments welcome. Assuming there are no strong objections asking for reversion of this change, I'll publicise to the wider community in a few days.
It strikes me as a solid, pragmatic solution to a thorny problem.
Looking at your checkin though, I wonder if it might be worth implementing some little formatting style classes to get rid of the if/elif chains from the Formatter code. Something like:
Some syntax highlighting may make that wall-o'-code a little easier to read: http://pastebin.com/SG0Qr3r5
Cheers, Nick.
-- Nick Coghlan | ncoghlan@gmail.com | Brisbane, Australia _______________________________________________ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/bostjan.mejak%40gmail.com
On Tue, Oct 26, 2010 at 6:15 AM, Nick Coghlan <ncoghlan@gmail.com> wrote:
On Tue, Oct 26, 2010 at 9:08 PM, Nick Coghlan <ncoghlan@gmail.com> wrote:
On Tue, Oct 26, 2010 at 12:28 AM, Vinay Sajip <vinay_sajip@yahoo.co.uk> wrote:
Comments welcome. Assuming there are no strong objections asking for reversion of this change, I'll publicise to the wider community in a few days.
It strikes me as a solid, pragmatic solution to a thorny problem.
Looking at your checkin though, I wonder if it might be worth implementing some little formatting style classes to get rid of the if/elif chains from the Formatter code. Something like:
Some syntax highlighting may make that wall-o'-code a little easier to read: http://pastebin.com/SG0Qr3r5
FWIW +1 -- Regards, Olemis. Blog ES: http://simelo-es.blogspot.com/ Blog EN: http://simelo-en.blogspot.com/ Featured article:
Nick Coghlan <ncoghlan <at> gmail.com> writes:
Looking at your checkin though, I wonder if it might be worth implementing some little formatting style classes to get rid of the if/elif chains from the Formatter code. Something like:
Fair comment: I did think about the messiness of that if/elif, but considered it OK as it was likely that Python would not grow any more formatting approaches. However, I hadn't thought about user-defined approaches; what you're suggesting is more extensible. I'll incorporate your and Boštjan's comments in the next checkin. Thanks, Vinay
On 10/26/10 7:08 AM, Nick Coghlan wrote:
On Tue, Oct 26, 2010 at 12:28 AM, Vinay Sajip<vinay_sajip@yahoo.co.uk> wrote:
Comments welcome. Assuming there are no strong objections asking for reversion of this change, I'll publicise to the wider community in a few days.
It strikes me as a solid, pragmatic solution to a thorny problem.
I keep meaning to review this but haven't had time. One thing I want to look at specifically is the ability to put the time formatting into the str.format version of the format string. Now that the time format specifier can be included in the format string, it's no longer necessary to have the asctime inspection hack that's currently used in order to avoid formatting the time. It would be good if we could remove the formatTime logic for str.format, although I'm not sure how practical it is. I suspect it's too baked-in to the design, but I'm hopeful. I'm +1 on the overall concept of allow the other string formatters. -- Eric.
Eric Smith <eric <at> trueblade.com> writes:
I keep meaning to review this but haven't had time. One thing I want to look at specifically is the ability to put the time formatting into the str.format version of the format string. Now that the time format specifier can be included in the format string, it's no longer necessary to have the asctime inspection hack that's currently used in order to avoid formatting the time. It would be good if we could remove the formatTime logic for str.format, although I'm not sure how practical it is. I suspect it's too baked-in to the design, but I'm hopeful.
Well, asctime checks would be in there for the other format styles anyway, and you don't gain anything by making str.format a special case where the check is avoided. The %-style will be around for a while, particularly as AFAICT {}-formatting is still nominally slower than %-formatting [it *is* a lot more flexible, so I can understand that]and people still cling to the "logging is slow" meme, despite there being no hard numbers presented in evidence. You don't have to specify asctime in the format string, but remember that since Python logging predates datetime [both came in at Python 2.3 but the logging package was independently available before, and compatible with Python 1.5.2 for a while after introduction], times in logging are floats as per time.time(). IIUC you need a datetime.datetime object to do the formatting automatically using str.format, so if one could e.g. use a Filter or LoggerAdapter to get a datetime value into the LogRecord, then you could use {}-formatting without {asctime} but with some other LogRecord attribute. I think the way it is now for 3.2 is the path of least resistance, though. BTW as it is now, the asctime "hack" is less of a hack than it used to be; it was an inline check before, but now can be easily reimplemented e.g. via subclassing and overriding the usesTime() method. Regards, Vinay Sajip
On Oct 25, 2010, at 02:28 PM, Vinay Sajip wrote:
I've just checked in a change to logging into the py3k branch (r85835), including doc changes and tests, for providing slightly more flexibility in alternative format styles for logging.
Basically, Formatter.__init__ gets an extra optional keyword arg style=<one of '%' (default), '{' or '$'>. This is then used to merge the format string with the LogRecord: either fmt % record.__dict__, or fmt.format(**record.dict), or string.Template(fmt).substitute(**record.dict). Backward compatibility is maintained (unless I've missed something).
This sounds like a reasonable solution that provides the flexibility we want, while maintaining backward compatibility. Thanks! I haven't played with it yet, but do you think it makes sense to add a 'style' keyword argument to basicConfig()? That would make it pretty easy to get the formatting style you want without having to explicitly instantiate a Formatter, at least for simple logging clients. Cheers, -Barry
On Fri, Oct 29, 2010 at 10:07 AM, Barry Warsaw <barry@python.org> wrote:
On Oct 25, 2010, at 02:28 PM, Vinay Sajip wrote:
I've just checked in a change to logging into the py3k branch (r85835), including doc changes and tests, for providing slightly more flexibility in alternative format styles for logging.
Basically, Formatter.__init__ gets an extra optional keyword arg style=<one of '%' (default), '{' or '$'>. This is then used to merge the format string with the LogRecord: either fmt % record.__dict__, or fmt.format(**record.dict), or string.Template(fmt).substitute(**record.dict). Backward compatibility is maintained (unless I've missed something).
This sounds like a reasonable solution that provides the flexibility we want, while maintaining backward compatibility. Thanks!
I haven't played with it yet, but do you think it makes sense to add a 'style' keyword argument to basicConfig()? That would make it pretty easy to get the formatting style you want without having to explicitly instantiate a Formatter, at least for simple logging clients.
Since this may be considered as a little sophisticated, I'd rather prefer these new classes to be added to configuration sections using fileConfig (and default behavior if missing), and still leave `basicConfig` unchanged (i.e. *basic*) . PS: Good work ! -- Regards, Olemis. Blog ES: http://simelo-es.blogspot.com/ Blog EN: http://simelo-en.blogspot.com/ Featured article:
Olemis Lang <olemis <at> gmail.com> writes:
On Fri, Oct 29, 2010 at 10:07 AM, Barry Warsaw <barry <at> python.org> wrote:
I haven't played with it yet, but do you think it makes sense to add a 'style' keyword argument to basicConfig()? That would make it pretty easy to get the formatting style you want without having to explicitly instantiate a Formatter, at least for simple logging clients.
Since this may be considered as a little sophisticated, I'd rather prefer these new classes to be added to configuration sections using fileConfig (and default behavior if missing), and still leave `basicConfig` unchanged (i.e. *basic*) .
Actually it's no biggie to have an optional style argument for basicConfig. People who don't use it don't have to specify it; the style argument would only apply if format was specified. For some people, use of {} over % is more about personal taste than about the actual usage of str.format's flexibility; we may as well accommodate that preference, as it encourages in a small way the use of {}-formatting. Regards, Vinay Sajip
On Sun, 31 Oct 2010 14:55:34 -0000, Vinay Sajip <vinay_sajip@yahoo.co.uk> wrote:
Olemis Lang <olemis <at> gmail.com> writes:
On Fri, Oct 29, 2010 at 10:07 AM, Barry Warsaw <barry <at> python.org> wrote:
I haven't played with it yet, but do you think it makes sense to add a 'style' keyword argument to basicConfig()? That would make it pretty easy to get the formatting style you want without having to explicitly instantiate a Formatter, at least for simple logging clients.
Since this may be considered as a little sophisticated, I'd rather prefer these new classes to be added to configuration sections using fileConfig (and default behavior if missing), and still leave `basicConfig` unchanged (i.e. *basic*) .
Actually it's no biggie to have an optional style argument for basicConfig. People who don't use it don't have to specify it; the style argument would only apply if format was specified.
For some people, use of {} over % is more about personal taste than about the actual usage of str.format's flexibility; we may as well accommodate that preference, as it encourages in a small way the use of {}-formatting.
+1 -- R. David Murray www.bitdance.com
On Sun, Oct 31, 2010 at 9:55 AM, Vinay Sajip <vinay_sajip@yahoo.co.uk> wrote:
Olemis Lang <olemis <at> gmail.com> writes:
On Fri, Oct 29, 2010 at 10:07 AM, Barry Warsaw <barry <at> python.org> wrote:
I haven't played with it yet, but do you think it makes sense to add a 'style' keyword argument to basicConfig()? That would make it pretty easy to get the formatting style you want without having to explicitly instantiate a Formatter, at least for simple logging clients.
Since this may be considered as a little sophisticated, I'd rather prefer these new classes to be added to configuration sections using fileConfig (and default behavior if missing), and still leave `basicConfig` unchanged (i.e. *basic*) .
Actually it's no biggie to have an optional style argument for basicConfig. People who don't use it don't have to specify it; the style argument would only apply if format was specified.
ok
For some people, use of {} over % is more about personal taste than about the actual usage of str.format's flexibility;
Thought you were talking about me, you only needed to say «he has black hair and blue eyes» ... ;o)
we may as well accommodate that preference, as it encourages in a small way the use of {}-formatting.
ok , nevermind , it's ok for me anyway (provided that sections for `fileConfig` will be available) . -- Regards, Olemis. Blog ES: http://simelo-es.blogspot.com/ Blog EN: http://simelo-en.blogspot.com/ Featured article:
Olemis Lang <olemis <at> gmail.com> writes:
For some people, use of {} over % is more about personal taste than about the actual usage of str.format's flexibility;
Thought you were talking about me, you only needed to say «he has black hair and blue eyes» ... ;o)
No, it was a general comment; I don't know your preferences. The basicConfig() change has now been checked into the py3k branch. Regards, Vinay Sajip
participants (7)
-
Barry Warsaw -
Boštjan Mejak -
Eric Smith -
Nick Coghlan -
Olemis Lang -
R. David Murray -
Vinay Sajip