Definitely use syslog.  To see how much logging is costing you without changing code, you can simply route to /dev/null via syslog.  Btw, w/ local logging, you may not see it in the cpu usage - more likely in the i/o wait.  In my experience, logging to a remote syslog-ng server over udp has very little impact on the app, almost like logging to /dev/null (which in turn is like not logging at all).  

Are you by chance on AWS/EC2 ?  If so, please contact me to discuss possible design partnership.  

m




On Fri, Apr 9, 2010 at 11:57 AM, Jason J. W. Williams <jasonjwwilliams@gmail.com> wrote:
Hi Paul,

Is is sporadic? (the CPU burst) If so, you may be seeing the log
rotation code kicking...and it's probably blocking your app too until
it completes.

I've got a couple patches to fix the blocking issue:

http://twistedmatrix.com/trac/ticket/4372

The patch which eliminates old log rotation will probably get rid of
your CPU problem too.

-J

On Fri, Apr 9, 2010 at 2:35 AM, Paul Goins <general@vultaire.net> wrote:
> There may not be a clear answer to this problem, and the answer may or
> may not be Twisted-specific.  I'm just trying to fish for ideas here.
>
> The server I'm working on suffers from extremely high CPU time spent in
> the logging functions.  I'm spending between 30 and 40% of all CPU time
> just in twisted.python.log.msg.
>
> Describing the environment:
>
> We use DailyLogFiles for file rotation along with the
> PythonLoggingObserver to allow for logLevel support.  (Of course, this
> gives us significant overhead from Python's complex logging
> architecture; maybe we should drop this...)
>
> On a 1000 request test of this server, we generate 58,274 calls to
> t.p.log.msg.  There's room for cleanup, but there's lots of pressure to
> keep many of the log calls we have for forensic purposes, so we can only
> cut this back so far.
>
> Anyway, does anyone have any ideas for speedups?  Any stories from
> experience especially?  Shots in the dark are welcome.  ;-)
>
> -
> Paul Goins
>
> _______________________________________________
> Twisted-Python mailing list
> Twisted-Python@twistedmatrix.com
> http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python
>

_______________________________________________
Twisted-Python mailing list
Twisted-Python@twistedmatrix.com
http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python