[stdlib-sig] logging

[Jesse Noller]

On Fri, Sep 18, 2009 at 4:43 AM, Vinay Sajip <vinay_sajip at yahoo.co.uk> wrote:
> Antoine Pitrou <solipsis at ...> writes:
>
>
>> Pylons uses it too.
>>
>> While logging may not look extremely pretty, I have yet to see another
>> logging library that has a pretty *and* powerful API. Applications /
>> libraries which reinvent their own logging API usually end up with
>> something which is both less powerful and *not* prettier (see
>> twisted.log for an example).
>>
>
> Thanks. If you had the time to write your ideal, "pretty" API - hypothetically,
> say, to wrap logging so you wouldn't use the underlying power - what would this
> API look like? I'm open to ideas from all of you.

Hi Vinay;

I've kept my mouth shut (well, on this subject) simply due to the fact
I tend to feel API design is a bit of a "smell" thing. First off;
thank you for the package. As much as I might not like the API - I use
the logging package an *insane* amount. I also know you're responsive,
and you care strongly about it. I myself might come off as slightly
defensive for my own module (multiprocessing) if someone where to just
say "lol it sux hahaha" (in fact, I have).

You are right; it is flexible, and it is meant for a wide-range of use
cases, and when you really "get it" it can be wildly powerful (I think
David Beazley wrote a twitter handler!).

However, my particular gut feeling when dealing with it stems from
something I can't quite communicate properly. I *want* something
"simpler" - for example, something which logs messages at a certain
level to stderr (but not stdout) and stdout to stdout (but not stderr)
- but also has a file logger.

I don't want to have to write more than a few lines of code to do this
- in *my* mind this is something so fundamental to
unixy-scripts/daemons than it should be as simple as:

import logging

log = logging.get_log('mylog')
log.warning('hay guys')

What I end up doing in most of my projects (sorry, not public ones) is
wrapping this in a "jesse.log" module that offers that API. The user
does not see the complexity of the underlying logging module's APIs.
In fact, I have a nasty tendency to create one "log" object which also
has a fair amount of the logging module's API pushed into it, e.g.:

    from jesse import log

    log = log()
    log.critical('yay!')
    log.set_level(log.WARNING)  # I loathe BouncyNames
    log.add_handler(log.file_handler(level=log.CRITICAL))


The other aspect of this is my experience trying to explain logging to
people who have never dealt with that module before. Recently, I was
trying to explain it to someone who has limited python knowledge, and
really just wanted something like what I describe above. They read the
docs, re-read the docs, re-re-re-read the docs, and still came to me
and said "how on earth do I do this?!". The API doesn't (and this
again, is a smell thing) feel python-y - it feels very Java like
(having experience log4j, I can say it really does feel like it). I
think this trips newbies up quite a bit.

Part of the newb-to-not-newb transition would be helped by possibly
simplifying the docs (something I am *still* working on for
multiprocessing) - the examples can be like drinking from a fire hose.
Doug Hellmann - the author of the Python Module of the Week is an
*excellent* doc writer (see his logging write up here:
http://www.doughellmann.com/PyMOTW/logging/index.html#module-logging)
and might be willing to help, give pointers, what have you.

Like I said at the start - this is all a "smell" thing, and it
obviously varies from person to person. This is fundamentally why I
was interested in encouraging Mishok and others to put together
concrete ideas together (I would be interested in seeing an
alternative implementation as a thought exercise). I know others
besides me have written little wrappers around logging, for example:

http://pypi.python.org/pypi/easylog/
http://pypi.python.org/pypi/autolog/
http://pypi.python.org/pypi/sensible/

Perhaps that's a good place to start - higher level
functions/methods/etc to "scale down" loggings perceived complexity? I
know I'm trying to do bits of that for multiprocessing.

Then of course, there's time for something completely different ;)

http://code.zacharyvoase.com/lumberjack/src/

jesse