[Python-Dev] Consistent logging in the standard library

[Matthew F. Barnes]

Guido van Rossum said:
> Hm...  Isn't the logging module's ability to set separate *handlers*
> sufficient to get the desired flexibility?  Rather than the
> application specifying a different *logger* object, the application
> can specify a different *handler* object.  I think this was the
> originally intended design.  It places less of a burden on the modules
> that use logging (no need to define getdefaultlogger() and
> setdefaultlogger()) and makes it possible for the different handlers
> to be specified by the logging module's configuration file, without
> the application having to support this.

I was thinking of the logging hierarchy when I came up with this idea.  If
a module references a specific logger object explicitly when a logging
message needs to be issued (e.g. logging.getLogger(__name__).info(msg)),
then the module's logging output is forever handled at a fixed place in
the hierarchy.

An application may want to organize things differently.  For example, it
may want to redirect asyncore's log messages to a logger named
"myapp.network.asyncore" so that its messages are propagated to the
"myapp.network" and "myapp" loggers defined by the application.  Those
loggers could apply additional handlers or filters to asyncore's log
messages.  In such a case the application could set this up as follows:

    asyncore.setdefaultlogger(logging.getLogger('myapp.network.asyncore'))


> What behavior exactly are you thinking of?  A logger object has almost
> no semantics apart from passing log events on to handlers; by design,
> all the work is done in the handlers.

I used the word "behavior" to refer to the series of handlers and filters
that are applied to a log message.  Different instances of the same class
could log to different logger objects, and thereby have different "logging
behaviors".  Unfortunately none of the modules I've identified in the
stdlib seem to provide a motivating example of why you'd ever want to do
this.  I can try to elaborate further if the benefit is not clear.


>> I determined that no changes are necessary for the cgi module, since
>> it doesn't even use the logging functions it defines (initlog,
>> dolog, etc.).
>
> They're intended for use by CGI applications.  They should probably be
> rewritten to direct the calls to the logging module.

I don't have much experience with this module so correct me if I'm wrong,
but to my knowledge the cgi module does not call its own logging
functions.  This implies to me that it's essentially up to the CGI
application to perform its own logging.  If that's true then I wonder if
CGI applications would be better off just be using the logging package
directly, rather than routing log messages through the cgi module.

The existing log functions in cgi should probably remain for backwards
compatibility, but I just have to wonder if it's really worth updating
them.


>> I'm currently working on a way to replicate imaplib's command
>> history using LogRecords, which may call for a new type of
>> logging.BufferingHandler subclass that maintains a circular queue of
>> LogRecords.
>
> That might be a useful feature on its own for the logging package.
> Note though, that at a grander scale, the RotatingFileLogger class
> already supports a way to keep only the last N messages (for much
> larger N, though :-).

Thanks, I'll look into that.


> To reach Steve, you should probably post to the PyUnit list (sorry,
> you'll have to Google for details, I only vaguely recall that there is
> such a list).

If Pyunit-interest is what you mean, the last time I checked it didn't
seem to be very active anymore and was filled with mostly spam.  I'll try
to reach Steve directly.


> Personally, I think that unittest is such a special case that it might
> be best to leave it alone...

I think I agree with you.  Unless Steve has any ideas, I'm going to drop
it from the forthcoming PEP.