[Chicago] Taking Some Pain Out Of Python Logging

[Philip Doctor]

Maybe a dumb question on my part, but can someone sort of clarify what the
exact problem with logging is?  The author states people hate it (then
cites a tweet) and claims it "reeks of NIH" without much justification.  I
guess I don't find the standard logging to be that difficult?  Is there
something that's specifically hard to do with it?

As justification for a switch to syslog he states I'll gain all kinds of
good things:

* Consistent time stamps all over the server
correct me if I'm wrong please, but asctime is pulling from time.time()
which is getting seconds since epoch from the OS right?  Shouldn't I
already have consistent time stamps all over the sever?  I haven't
personally noticed any inconsistencies, but I'm admittedly usually only
looking at my application logs.

* Filtering into files by application
This confuses me also, when I configure a logger in an application, I can
also tell it to us a specific file for that application.  I'm not seeing a
win here.

*Log rotation
He might have a point there regarding compression of old logs etc

* Remote logging
I guess I use splunk for this so I do see a big win for me, but I could see
it as a win for some scenarios.

Any replies and horror stories from the world of python logging are
welcome.  I'd love to know what problems might be waiting for me up the
road as I'm currently using the default logging.







On Thu, Mar 7, 2013 at 9:13 AM, Peter Fein <pete at wearpants.org> wrote:

> On Thu, Mar 7, 2013 at 9:01 AM, Skip Montanaro <skip at pobox.com> wrote:
> > At work we've used the stdlib logger in the platform I work on for
> > quite awhile.  We log heavily, and were always disappointed in logging
> > performance.  It was mostly caused by all the features baked into the
> > stdlib logger which we never used (I'm sure inherited from log4j or
> > whatever its logical parent was).  Over time we tried various things,
> > including inlining a stripped down __init__, threading the loggers,
> > rewriting things in C++, yadda, yadda, yadda.  A few months ago I
> > broke down and wrote a very simple class which implements the basic
> > logging API, can only log in one format, and only uses those features
> > we actually need.  The platform is very mature, so I'm not worried
>
> Code please? :-)
>
> > (*) When you think about it, running a logger in its own thread is
> > probably not that useful.  The OS probably does a much better job of
> > scheduling writes to disk, and to switch threads you have to go
> > through the kernel anyway. The threaded logger has more trips through
> > the kernel to write to the file, once to switch to the logger thread,
> > once to write the bits. Just write the damn message and be done with
> > it...
>
> In my logger Twiggy, I have support for what I call "asynchronous
> logging" - use a child process to do the actual disk write.
> http://twiggy.wearpants.org/
>
> --
> Peter Fein | wearpants.org | @wearpants
>
> I read email at the start and end of each day. IM if urgent.
> _______________________________________________
> Chicago mailing list
> Chicago at python.org
> http://mail.python.org/mailman/listinfo/chicago
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/chicago/attachments/20130307/39e76422/attachment.html>