[Patches] [ python-Patches-804180 ] A ForwardingHandler for logging

[SourceForge.net]

Patches item #804180, was opened at 2003-09-11 00:07
Message generated for change (Settings changed) made by rhettinger
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=305470&aid=804180&group_id=5470

Category: Library (Lib)
Group: Python 2.4
Status: Open
Resolution: None
Priority: 5
Submitted By: Matthew F. Barnes (mfbarnes)
>Assigned to: Vinay Sajip (vsajip)
Summary: A ForwardingHandler for logging

Initial Comment:
This was suggested recently on python-dev.  I'm
proposing it as an addition to the standard set of
handlers in the logging package.  ForwardingHandler
simply forwards logging records to another logger
elsewhere in the logging hierarchy.  I like to think of
this handler as a "softlink" for logging.

The initialization method accepts the name of the
logger object it should forward to, which is then
fetched using getLogger().  This is preferrable to
accepting the logger object itself because it's easier
for a configuration file to reference another logger by
its "qualname".  If a name is not given to the
initialization method, the handler will forward to the
root logger.

The emit method guards against infinite loops that
would result from a logger object forwarding to itself
(perhaps the result of misconfiguration).

The attached file is not a "diff" file but just a code
fragment that can be inserted somewhere in
logging/__init__.py  (or in logging/handlers.py with
some minor modifications).

----------------------------------------------------------------------

Comment By: Vinay Sajip (vsajip)
Date: 2003-09-12 08:47

Message:
Logged In: YES 
user_id=308438

Whoops, mea culpa. I typed "logger" when I meant "handler". 
Sorry to have caused confusion.

----------------------------------------------------------------------

Comment By: Matthew F. Barnes (mfbarnes)
Date: 2003-09-12 07:42

Message:
Logged In: YES 
user_id=843448

Okay, perhaps I was misinterpreting the third item from your
previous comment where you said:

"But if I want to see it interspersed with other stuff I'm
doing in "myapp.network.asyncore", an application built on
top of asyncore, I just add the same logger to "asyncore"
and "myapp.network.asyncore",and I can see both sets of
events together."

I guess it's not clear to me what you mean by "the same
logger".  Perhaps you meant "the same handler", which would
be more in line with your most recent comments?

----------------------------------------------------------------------

Comment By: Vinay Sajip (vsajip)
Date: 2003-09-12 03:19

Message:
Logged In: YES 
user_id=308438

I think/hope I understand the issue you're trying to address, 
and I think the existing API supports your use case. But your 
comment

"The idea of using the same logger object at multiple points in 
the hierarchy is interesting and had not occurred to me"

appears to mix up loggers and handlers. It's handlers which 
are attachable to multiple points in the hierarchy - loggers 
define the hierarchy. Loggers relate to events which occur in 
code - it's the developer's choice of logger name which 
indicates where in their scheme of things an event belongs. 
However, the audience of interested parties for the event is 
dynamic, and for that you need to use handlers, not loggers. 
To achieve the effect where you want a library module's 
logging output to be directed to your application's log, which 
is the use case you mention, add an appropriate handler (e.g. 
one which writes to your application's disk log file) to the 
module's logger. So, if a module has a logger accessible via a 
getCurrentLogger() function, you can say

myAppLogFileHandler = FileHandler(...)

module.getCurrentLogger().addHandler(myAppLogFileHandler)

and then the module's events get logged to your application's 
log file. They will appear with the logger name specified by 
the module, making it very clear where the event originated.



----------------------------------------------------------------------

Comment By: Matthew F. Barnes (mfbarnes)
Date: 2003-09-12 00:33

Message:
Logged In: YES 
user_id=843448

The patch was motivated by an issue I see where a module has
hard-coded a logger name into its logic, and an application
would like to use that module as-is (e.g. a module from the
standard library) but direct log messages from that module
elsewhere in the logging hierarchy.

The ForwardingHandler is intended to be a mechanism for
doing so.  The idea of using the same logger object at
multiple points in the hierarchy is interesting and had not
occurred to me, but it's not clear to me from the Python 2.3
documentation how to set that up (neither via the logging
API nor from a configuration file), and I'm not sure if it
addresses the issue I'm trying to resolve.

You did identify some legitimate bugs in comments (1) and
(2), and I've submitted a second patch which hopefully
should address them.  The bugs I've tried to address are
defaulting to root, comparing a Handler instance to a Logger
instance, and having the wrong logger name show up in the
recipient's output.


----------------------------------------------------------------------

Comment By: Matthew F. Barnes (mfbarnes)
Date: 2003-09-12 00:32

Message:
Logged In: YES 
user_id=843448

The patch was motivated by an issue I see where a module has
hard-coded a logger name into its logic, and an application
would like to use that module as-is (e.g. a module from the
standard library) but direct log messages from that module
elsewhere in the logging hierarchy.

The ForwardingHandler is intended to be a mechanism for
doing so.  The idea of using the same logger object at
multiple points in the hierarchy is interesting and had not
occurred to me, but it's not clear to me from the Python 2.3
documentation how to set that up (neither via the logging
API nor from a configuration file), and I'm not sure if it
addresses the issue I'm trying to resolve.

You did identify some legitimate bugs in comments (1) and
(2), and I've submitted a second patch which hopefully
should address them.  The bugs I've tried to address are
defaulting to root, comparing a Handler instance to a Logger
instance, and having the wrong logger name show up in the
recipient's output.


----------------------------------------------------------------------

Comment By: Vinay Sajip (vsajip)
Date: 2003-09-11 20:06

Message:
Logged In: YES 
user_id=308438

I would make the following comments about this patch:

1. The logger name captures what happened/where it 
happened. If an event gets forwarded to a different logger, it 
just gets handled by that logger's handlers (assuming filtering 
lets it through), but is otherwise unchanged. If you forward 
an event from "asyncore" to "myapp.network.asyncore", the 
name would not change - it would get passed 
to "myapp.network.asyncore" handlers but still get emitted 
with a name of "asyncore". The same effect could be 
achieved by configuring handlers appropriately. Doing it this 
way, you could quite get easily get duplicated messages, 
which makes logs less useful. Particularly, the defaulting to 
the root logger is not a good idea - since many events filter 
up to the root logger, it would be very easy with the default 
to get lots of duplicated messages.

2. In ForwardingHandler.emit, self is a ForwardingHandler but 
self.recipient is a Logger. So the "if" condition will always 
evaluate to true.

3. Event loggers are the equivalent of "publish", and event 
handlers the equivalent of "subscribe". Mixing the two is not 
necessarily such a good idea. Events should be logged, by the 
developer, using a logger name which makes sense in the 
context of the module, subsystem or application. So, if an 
event occurs in "asyncore", we want to know that it 
happened in "asyncore". But if I want to see it interspersed 
with other stuff I'm doing in "myapp.network.asyncore", an 
application built on top of asyncore, I just add the same 
logger to "asyncore" and "myapp.network.asyncore", and I 
can see both sets of events together. 

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=305470&aid=804180&group_id=5470