lighter weight options to Python's logging package?

[Vinay Sajip]

> Trent> Do you have any profile information for where in the logging
>     Trent> package the time is being spent?
>
> Looking back at a recent run from a few days ago Formatter.formatTime()
> seems to be a current problem.

As Kent suggested, don't use %(asctime)s if you don't want fancy time
formatting. If needed, you can use %(created)f and postprocess logs if
you want absolute times.

> We create our own LogRecord subclass.  Because that was such a heavyweight
> operation, instead of calling logging.LogRecord.__init__ from our class's
> constructor, I wound up inlining it and deleting the state it set that we
> never used.
>
>     Trent> I don't know how much of a perf issue it is for you. Perhaps
>     Trent> tweaking your usage of the logging package (perhaps using custom
>     Trent> Loggers, Filters, outputters, etc.)  could be made "good enough".
>
> We've done a bit already.  I will see if there's more we can do.  Our use of
> the package is actually very simple.  All we do are log basic messages.  Our
> Formatter instance includes the time, level and message.  That's it.  We
> have no need for displaying exception info, don't do threads, don't mess
> with the file or module names, etc.  The only thing we do that isn't
> supported directly by the package is to use a compressing version of the
> RotatingFileHandler class.

If you set logging._srcfile to None, there's no walking of the stack
done by the logging package to get caller info such as line no, func
name, file name etc.

Exception info is only logged if you set exc_info to 1 in a logging
call, or if you use the exception(...) method or module-level function.

I've just checked in a change into Subversion which adds two module
variables, logThreads and logProcesses (both set to 1 to enable current
behaviour). If you don't want to log thread info, set logThreads to 0
in the svn trunk version. If you don't want os.getpid() called, set
logProcesses to 0 in this version.

To summarise: for the version in the svn trunk, setting _srcFile to
None and logThreads and logProcesses to 0, and avoiding %(asctime)s in
your Formatter specification, should remove a fair chunk of the
overhead.

Regards,

Vinay Sajip