From logging to files to a better solution: syslog, Sentry, Logstash, ....
Thomas Güttler
hv at tbz-pariv.de
Fri Sep 11 03:22:22 EDT 2015
Am Donnerstag, 10. September 2015 08:42:47 UTC+2 schrieb dieter:
> Thomas Güttler writes:
> > ...
> > Why we are unhappy with logging to files:
> >
> > - filtering: We don't want to get INFO messages over the VPN.
>
> You can quite easily control at what level messages are logged with
> the standard Python logging framework. Each handler has a level
> and will ignore messages at a lower level.
I want INFO to be logged and stored on the remote host.
Therefore I must not filter INFO messages.
I don't want to pull INFO messages over the VPN.
Ergo, the filtering at Python level does not help in my use case.
Or I am missing something.
And now I have an ugly soup.
The ugly soup is a text file with not well defined syntax. It looks line
based. But sometimes there are log messages which span multiple lines ....
Remember: "Life is too short to (re)write parsers"
Yes, there are tools to parse that soup. But why create this soup in
the first place?
That's why I want to move from file based logging to a different solution.
Unfortunately there too many choices (graylog, logstash, sentry, ...) :-(
> > - Rotating: Rotating files is possible, but somehow cumbersome.
>
> There are standard tools to rotate logfiles.
>
> > - Support structured logging of values (json) in the future.
>
> Again, the Python logging framework is quite flexible with
> respect to the format of logged messages.
>
> > ...
> > Which solution could fit for our environment?
>
> I work for a customer with a similar environment (he uses "Zope" instead
> of "Django") - and he uses logfiles. The logfiles are automatically
> rotated and there are in the order of half a dozen to a dozen logfiles
> per day.
>
> When I have to analyse a problem with the help of the logfiles,
> I do not copy them via VPN but do the filtering remotely and only
> copy the filtered portion, if necessary.
Good to know that I am not the only one running servers in remote intranets.
Regards,
Thomas Güttler
More information about the Python-list
mailing list