[Patches] [ python-Patches-804180 ] A ForwardingHandler for logging

Fri Sep 12 06:42:03 EDT 2003

Patches item #804180, was opened at 2003-09-11 00:07
Message generated for change (Comment added) made by mfbarnes
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=305470&aid=804180&group_id=5470

Category: Library (Lib)
Group: Python 2.4
Status: Open
Resolution: None
Priority: 5
Submitted By: Matthew F. Barnes (mfbarnes)
Assigned to: Raymond Hettinger (rhettinger)
Summary: A ForwardingHandler for logging

Initial Comment:
This was suggested recently on python-dev.  I'm

proposing it as an addition to the standard set of

handlers in the logging package.  ForwardingHandler

simply forwards logging records to another logger

elsewhere in the logging hierarchy.  I like to think of

this handler as a "softlink" for logging.

The initialization method accepts the name of the

logger object it should forward to, which is then

fetched using getLogger().  This is preferrable to

accepting the logger object itself because it's easier

for a configuration file to reference another logger by

its "qualname".  If a name is not given to the

initialization method, the handler will forward to the

root logger.

The emit method guards against infinite loops that

would result from a logger object forwarding to itself

(perhaps the result of misconfiguration).

The attached file is not a "diff" file but just a code

fragment that can be inserted somewhere in

logging/__init__.py  (or in logging/handlers.py with

some minor modifications).

----------------------------------------------------------------------

>Comment By: Matthew F. Barnes (mfbarnes)
Date: 2003-09-12 07:42

Message:
Logged In: YES 
user_id=843448

Okay, perhaps I was misinterpreting the third item from your

previous comment where you said:

"But if I want to see it interspersed with other stuff I'm

doing in "myapp.network.asyncore", an application built on

top of asyncore, I just add the same logger to "asyncore"

and "myapp.network.asyncore",and I can see both sets of

events together."

I guess it's not clear to me what you mean by "the same

logger".  Perhaps you meant "the same handler", which would

be more in line with your most recent comments?

----------------------------------------------------------------------

Comment By: Vinay Sajip (vsajip)
Date: 2003-09-12 03:19

Message:
Logged In: YES 
user_id=308438

I think/hope I understand the issue you're trying to address, 

and I think the existing API supports your use case. But your 

comment

"The idea of using the same logger object at multiple points in 

the hierarchy is interesting and had not occurred to me"

appears to mix up loggers and handlers. It's handlers which 

are attachable to multiple points in the hierarchy - loggers 

define the hierarchy. Loggers relate to events which occur in 

code - it's the developer's choice of logger name which 

indicates where in their scheme of things an event belongs. 

However, the audience of interested parties for the event is 

dynamic, and for that you need to use handlers, not loggers. 

To achieve the effect where you want a library module's 

logging output to be directed to your application's log, which 

is the use case you mention, add an appropriate handler (e.g. 

one which writes to your application's disk log file) to the 

module's logger. So, if a module has a logger accessible via a 

getCurrentLogger() function, you can say

myAppLogFileHandler = FileHandler(...)

module.getCurrentLogger().addHandler(myAppLogFileHandler)

and then the module's events get logged to your application's 

log file. They will appear with the logger name specified by 

the module, making it very clear where the event originated.

----------------------------------------------------------------------

Comment By: Matthew F. Barnes (mfbarnes)
Date: 2003-09-12 00:33

Message:
Logged In: YES 
user_id=843448

The patch was motivated by an issue I see where a module has

hard-coded a logger name into its logic, and an application

would like to use that module as-is (e.g. a module from the

standard library) but direct log messages from that module

elsewhere in the logging hierarchy.

The ForwardingHandler is intended to be a mechanism for

doing so.  The idea of using the same logger object at

multiple points in the hierarchy is interesting and had not

occurred to me, but it's not clear to me from the Python 2.3

documentation how to set that up (neither via the logging

API nor from a configuration file), and I'm not sure if it

addresses the issue I'm trying to resolve.

You did identify some legitimate bugs in comments (1) and

(2), and I've submitted a second patch which hopefully

should address them.  The bugs I've tried to address are

defaulting to root, comparing a Handler instance to a Logger

instance, and having the wrong logger name show up in the

recipient's output.

----------------------------------------------------------------------

Comment By: Matthew F. Barnes (mfbarnes)
Date: 2003-09-12 00:32

Message:
Logged In: YES 
user_id=843448

The patch was motivated by an issue I see where a module has

hard-coded a logger name into its logic, and an application

would like to use that module as-is (e.g. a module from the

standard library) but direct log messages from that module

elsewhere in the logging hierarchy.

The ForwardingHandler is intended to be a mechanism for

doing so.  The idea of using the same logger object at

multiple points in the hierarchy is interesting and had not

occurred to me, but it's not clear to me from the Python 2.3

documentation how to set that up (neither via the logging

API nor from a configuration file), and I'm not sure if it

addresses the issue I'm trying to resolve.

You did identify some legitimate bugs in comments (1) and

(2), and I've submitted a second patch which hopefully

should address them.  The bugs I've tried to address are

defaulting to root, comparing a Handler instance to a Logger

instance, and having the wrong logger name show up in the

recipient's output.

----------------------------------------------------------------------

Comment By: Vinay Sajip (vsajip)
Date: 2003-09-11 20:06

Message:
Logged In: YES 
user_id=308438

I would make the following comments about this patch:

1. The logger name captures what happened/where it 

happened. If an event gets forwarded to a different logger, it 

just gets handled by that logger's handlers (assuming filtering 

lets it through), but is otherwise unchanged. If you forward 

an event from "asyncore" to "myapp.network.asyncore", the 

name would not change - it would get passed 

to "myapp.network.asyncore" handlers but still get emitted 

with a name of "asyncore". The same effect could be 

achieved by configuring handlers appropriately. Doing it this 

way, you could quite get easily get duplicated messages, 

which makes logs less useful. Particularly, the defaulting to 

the root logger is not a good idea - since many events filter 

up to the root logger, it would be very easy with the default 

to get lots of duplicated messages.

2. In ForwardingHandler.emit, self is a ForwardingHandler but 

self.recipient is a Logger. So the "if" condition will always 

evaluate to true.

3. Event loggers are the equivalent of "publish", and event 

handlers the equivalent of "subscribe". Mixing the two is not 

necessarily such a good idea. Events should be logged, by the 

developer, using a logger name which makes sense in the 

context of the module, subsystem or application. So, if an 

event occurs in "asyncore", we want to know that it 

happened in "asyncore". But if I want to see it interspersed 

with other stuff I'm doing in "myapp.network.asyncore", an 

application built on top of asyncore, I just add the same 

logger to "asyncore" and "myapp.network.asyncore", and I 

can see both sets of events together. 

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=305470&aid=804180&group_id=5470