[Python-Dev] Logging enhancements

[Shane Hathaway]

Vinay Sajip wrote:
> On 1/9/06, Shane Hathaway <shane at hathawaymix.org> wrote:
>>- The logging.config.fileConfig function has bare excepts that hide
>>configuration errors.  I'd like to remove the bare excepts and break up
>>fileConfig for clarity.
> 
> I've no problem with this in principle, though there have been cases where
> ConfigParser behaviour is not what you would expect (e.g. a non-existent
> file leads to a "missing section" exception). I'd be grateful for more
> specifics as to where you have encountered problems.

I wrote a config file with an invalid handler section.  Because of the 
bare except, not only was my error not reported anywhere, but the daemon 
started with no logging at all.  It took me a while to realize what had 
happened.  The logging package needs to report erroneous configurations.

>>- I'd like to make it possible to configure the format of exception
>>tracebacks.  Some logs want compressed tracebacks like Medusa; others
>>want extra information like Zope.
> 
> Again, no problem in principle, but it's probably an unusual use case so any
> change should not require extra work for users without specialised
> requirements.

Agreed.

>>- There is no support for external log rotation.  There is plenty of
>>support for Python rotating logs directly, but permission settings often
>>make that impossible.  Long-running Python processes need to accept a
>>signal which notifies them when an external tool rotates the logs.
> 
> I'd be interested in your approach to solve this.

To be compatible with an external tool like logrotate, all a daemon 
needs to do is close and reopen log files at about the same time log 
rotation happens.  To handle this use case, I suggest the logging module 
needs a function named reopen(), which daemons can call upon receipt of 
a signal.  The reopen function will find all handlers with open files 
and ask them to reopen.

>>- The logging documentation is lacking somewhat.  For example, it fails
>>to mention that the logging handlers are in the 'handlers' module, which
>>I only learned by searching the source code.
> 
> 
> Not true for this specific case - for example,
> http://docs.python.org/lib/module-logging.html has a list of 12 handlers,
> followed by a statement to the effect that all but StreamHandler and
> FileHandler are defined in the handlers module.

Oops.  I guess I skipped over that because I was using the documentation 
as a reference rather than as a tutorial.  I suggest the documentation 
needs to become a better reference.

> In general, patches are gratefully received! If you don't want to invest too
> much time up front, send me a mail indicating your general approach and
> we'll discuss.

I'll get started soon.  I just wanted to be sure I'm not duplicating 
someone else's efforts.

Shane