[Baypiggies] Python Logging Module

steve hindle mech422 at gmail.com
Thu May 18 14:33:01 CEST 2006

On 5/18/06, hazmat at objectrealms.net <hazmat at objectrealms.net> wrote:
> the logging module was designed to be pluggable.. and all of these
> issues can be solved around via the same.
Thanks - but that wasn't what I asked.  Since nobody responded with a
'canned' solution, I went ahead and rolled my own.  I've responded to
your comments below

>   write a custom logger that overrides the _log method, use a custom log
> record class which captures this information (its currently gotten and
> discarded in the base handler's _log method), and a formatter which
> exposes it.
Yeah - its a commonly known (and solved) problem.  However, it seems a
pretty basic oversight for a logging module...

> > 2.) logging.config _disables_ all existing loggers when loading a new
> > config file
> >      Can't break logging configs down into small, bite-sized pieces
> >      (e.g. 1 file per module, 1 file per script, etc.)
This is a big one for me - I don't really like a 'loading information'
operation turning into a
'silently disable anything not in the new configfile' operation.
Maybe it just me, but I would have preferred something like
'logging.clear()' to disable loggers.  Then you could either add new
loggers, or clear() and totally reload the logging config.

> then, don't use that the default configuration. nothing forcing you to..
> its straight forward to write your own. i wrote a basic config
> alternative a few years ago to add a log profile abstraction for runtime
> switching of logging configuration and doing log configuration in python.

Again, this isn't really logging's fault - its a hold over from
configParser, but its still nasty.  Given the 'batteries included'
policy of Python, it would be nice to have a better configuration
parser in the stdlib.  The configObj module is a bit nicer, but still
requires multiple '[' ']' pairs to indicate section nesting.

> http://svn.objectrealms.net/view/occams/versioning/trunk/log.py?rev=164&view=auto
I'll have to take a look at this.. nice to see another Zopista about
:-)  LOL - I just realized where I've seen your nick - your hazmat
from #plone ?

> another example.. zope uses zconfig (standalone pluggable apache style
> configuration syntax for python apps) for its log handling which is
> included in the zconfig package.
This is really the sort of thing I was after - I'll have to check it
out.  Wish I'd seen it before hacking my own stuff in :-(

> > 3.) Can't _STOP_ propagating a log message after it's handled
> >      'propagate' has no concept of 'propagate until someone deals with this'.
> >      This results in my top-level, 'catch-all' logger logging all message - even
> >      if they have already been proccessed
> >
> the logging module already provides a facility for this.. it stops
> propogating when it encounters a logger with the propagate flag set to false
propagate is very odd, and useless for any kind of 'catch-all'
processing.  If you want a top level logger to 'just deal with'
anything that percolates up, propagate has to be set on every logger
in the chain.  The result is that the 'catch-all' is _always_ invoked,
causing msgs to be logged multiple times.  I just hung an 'emitted'
attribute off the LogRecord, and a 'reEmit' off the logger and use
those to control whether processed messages are re-raised.
  I find the behavior counter intuative.  Why it has exactly the
opposite behavior of exception handling is beyond me (exceptions need
to be explicitly re-raised, but loggers 'propagate' implicitly?? seems
sorta inconsistent).

> or you could define a callhandlers method implementation in your custom
> handler, that doesn't propogate.

yeah did that - still seems like something that should be there by
default though

> the logging module has a lot of functionality, which duplicating would
> be a waste of effort imho unless its completely unsuitable for your
> requirements, working to improve it or using its pluggable nature is to
> the benefit of all. batteries included, does not mean every desire
> satisfied. that said, out side of #4, all of these sound nice for
> a patch for the logging module, though changing #3 has some behavioral
> compatibility issues.

Oh, I'm not about to re-write from scratch - I took the subclass and
monkey patch approach.  It just seemed like the logging stuff looked
better on paper then it worked out in practice.  For instance, a root
logger is created on import of the logging module - BEFORE you can
subclass Logger, so you either have to muck about getting a new root
logger in place or live with Logger.  If there was a real Init()
function that created the root logger, etc. You could just call that
AFTER you call setLoggerClass.

Anyway, I'll have a look at ZConfig.  I've got the %(func) tag in
place, and the re-emit stuff.  But I haven't done anything about the
config file syntax yet.  Maybe I'll get lucky and just be able to use
ZConfig across the board and drop configObj entirely.  I'm not gonna
bother about the logging.config stuff - I'll just bypass it and create
the formatters, handlers and loggers by hand.


More information about the Baypiggies mailing list