logging via SocketHandler and TCPserver

[Larry Bates]

Every time I look at the logging module (up until now) I've given up and 
continue to use my home-grown logger that I've been using for years.   I'm not 
giving up this time ;-)

I find that I REALLY need to be able to monitor LOTS of running 
programs/processes and thought it would be nice to have them use SocketHandler 
logging and then I would write TCPServer to accept the log messages for 
real-time monitoring.  I Googled (is that now a verb?) for several hours and 
came up with some code that I've turned in to something that works, but I can't 
figure out how to disconnect the server once it is connected  The goal is to be 
able to start TCPServer, monitor the log messages sent via SocketHandler logger, 
disconnect, and move to the next application.  Eventually I would like to wrap a 
GUI around all of this for monitoring a complex web application.

Everything works, it just appears that I get into the while loop in
LogRecordStreamHandler.handle and it never breaks out (until I kill the client).
I can't seem to do anything with the LogRecordSocketReceiver.abort attribute to 
make it quit.

I'm sure it is something simple (stupid?), but I've spent about 4 hours and I'm 
not getting anywhere.

Thanks in advance for any assistance.

Regards,
Larry


Below is my code:

import sys
import time
import logging

if sys.argv[1] == 'client':
     import logging.config

     logging.config.fileConfig("logging.conf")

     #create logger
     logger = logging.getLogger("VESconsole")

     while 1:
         logger.debug("debug message")
         logger.info("info message")
         logger.warn("warn message")
         logger.error("error message")
         logger.critical("critical message")
         time.sleep(2)

elif sys.argv[1] == 'server':
     import cPickle
     import logging.handlers
     import SocketServer
     import struct
     import signal

     class LogRecordStreamHandler(SocketServer.StreamRequestHandler):
         """Handler for a streaming logging request.

         This basically logs the record using whatever logging policy is
         configured locally.
         """

         def handle(self):
             """
             Handle multiple requests - each expected to be a 4-byte length,
             followed by the LogRecord in pickle format. Logs the record
             according to whatever policy is configured locally.
             """
             while 1:
                 chunk = self.connection.recv(4)
                 if len(chunk) < 4:
                     break

                 slen = struct.unpack(">L", chunk)[0]
                 chunk = self.connection.recv(slen)
                 while len(chunk) < slen:
                     chunk = chunk + self.connection.recv(slen - len(chunk))

                 obj = self.unPickle(chunk)
                 record = logging.makeLogRecord(obj)
                 self.handleLogRecord(record)

         def unPickle(self, data):
             return cPickle.loads(data)

         def handleLogRecord(self, record):
             t = time.strftime('%a, %d %b %y %H:%M:%S',
                               time.localtime(record.created))

             print "%s %s" % (t, record.getMessage())

     class LogRecordSocketReceiver(SocketServer.ThreadingTCPServer):
         """simple TCP socket-based logging receiver suitable for testing.
         """

         allow_reuse_address = 1

         def __init__(self, host='localhost',
                      port=logging.handlers.DEFAULT_TCP_LOGGING_PORT,
                      handler=LogRecordStreamHandler):

             SocketServer.ThreadingTCPServer.__init__(self,
                                                      (host, port),
                                                      handler)
             self.abort = 0
             self.timeout = 1
             self.logname = None

         def serve_until_stopped(self):
             import select
             abort = 0
             while not abort:
                 rd, wr, ex = select.select([self.socket.fileno()],
                                            [], [],
                                            self.timeout)
                 if rd:
                     self.handle_request()

                 abort = self.abort

             print "serve_until_stopped exiting"

     #
     # Start ThreadingTCPServer instance to accept SocketHandler log
     # messages from client.
     #
     tcpserver = LogRecordSocketReceiver()
     print "Starting ThreadingTCPServer..."
     tcpserver.serve_until_stopped()

'''
#-----logging.conf-----
[loggers]
keys=root

[handlers]
keys=socketHandler

[formatters]
keys=simpleFormatter

[logger_root]
level=DEBUG
handlers=socketHandler

[handler_socketHandler]
class=handlers.SocketHandler
level=DEBUG
args=('localhost', handlers.DEFAULT_TCP_LOGGING_PORT)
host=localhost
port=DEFAULT_TCP_LOGGING_PORT

[formatter_simpleFormatter]
format=%(asctime)s - %(name)s - %(levelname)s - %(message)s
datefmt=
'''