On Fri, 2010-04-09 at 15:23 +0200, Itamar Turner-Trauring wrote:
On Fri, 2010-04-09 at 11:49 +0200, Kees Bos wrote:
I think you could try to use a custom light weight log function that just queues the log messages (FIFO) and bursts them, say every second, to the log file (e.g. in a thread to use a multicore cpu).
Or:
0. Profile logging system - what exactly is it doing that's using all that CPU? If you're unlucky, it's just "lots of Python function calls", but maybe it's fixable.
1. A log function that sends messages (over e.g. a Unix socket) to another process that does the writing. Python GIL means you don't get that much benefit from multiple threads. Duh. Of course.
BTW. syslog from the syslog module on linux just opens a unix socket to /dev/log and could be used to do just this. (e.g. with log level LOG_LOCAL0 and configuring your syslog to log facility local0 to a specific file)