On Thu, Jun 13, 2013 at 5:14 PM, Glyph email@example.com wrote:
Hello everybody. Today I'd like to talk about everyone's favorite subsystem within Twisted: logging.
There are a number of problems with Twisted's logging system.
Thanks for writing up this email. I'd like to help out with this effort.
For a long time, there's been consensus among at least a few core Twisted developers that log messages ought to be specified in terms of a format message and several relevant keyword arguments, so that tools can extract the values of the keyword arguments without parsing ad-hoc messages or using regexes. In other words, like this:
log.msg(format="Foo happened. Relevant information includes: " "bar=%(bar)s baz=%(baz)s boz=%(boz)s", bar=bar, baz=baz, boz=boz)
There's something that I want to.... not propose, but discuss, that I've discovered through my writing of actual applications that do actual log filtering.
Basically, the conclusion I've come to is that English isn't very useful in logs. It's usually badly written, uninformative, and hard to filter on. The main thing that I find really important is having what I call an "event name", which has the following properties:
1. human readable 2. computer readable 3. mandatory 4. unique across a codebase, and specific to the physical logging statement that it's being passed to.
So, for example:
def frag_message_received(self, fragger, fragee, weapon): self.log("frag", fragger=fragger, fraggee=fragee, weapon=weapon) ...
Now, your smart log filterer can just filter by event="frag" instead of message="Frag for \w+ received with weapon \w+" or whatever.
This is just how I've done things in practice, and it has worked very well for fairly large projects, but there are improvements that I can think of. For example, the event name should probably be based on a hierarchy of composable logger objects (kinda like the one David Reid wrote) so that it's easier to maintain uniqueness in a large codebase.
(Hey David, you should write an email about that logging class :)