[Twisted-Python] Log rotates not as expected
Hi, in code below I try to rotate log files if they reach 5000000 limit, but this happens only to every 5 files, ls -althr: -rw-r--r-- 1 root root 977K 2010-01-19 19:03 /var/log/my.log.12 -rw-r--r-- 1 root root 977K 2010-01-19 19:55 /var/log/my.log.10 -rw-r--r-- 1 root root 977K 2010-01-19 20:41 /var/log/my.log.9 -rw-r--r-- 1 root root 977K 2010-01-19 21:30 /var/log/my.log.8 -rw-r--r-- 1 root root 977K 2010-01-19 22:46 /var/log/my.log.7 -rw-r--r-- 1 root root 4.8M 2010-01-20 00:19 /var/log/my.log.11 -rw-r--r-- 1 root root 977K 2010-01-20 00:19 /var/log/my.log.6 -rw-r--r-- 1 root root 977K 2010-01-20 09:17 /var/log/my.log.4 -rw-r--r-- 1 root root 977K 2010-01-20 10:45 /var/log/my.log.3 -rw-r--r-- 1 root root 977K 2010-01-20 12:04 /var/log/my.log.2 -rw-r--r-- 1 root root 977K 2010-01-20 12:59 /var/log/my.log.1 -rw-r--r-- 1 root root 4.2M 2010-01-20 13:26 /var/log/my.log.5 -rw-r--r-- 1 root root 301K 2010-01-20 13:26 /var/log/my.log it writes simultaneously to both my.log.5 and my.log. It is not big problem, but in this way I have only recent files, because they grows quickly Pet from twisted.python import log from twisted.python import logfile from twisted.application import service class MyLog(log.FileLogObserver): def emit(self, logEntryDict): log.FileLogObserver.timeFormat = '%Y-%m-%d %H:%M:%S' log.FileLogObserver.emit(self, logEntryDict) class MyLogService(service.Service): def __init__(self, logName, logDir): self.logName = logName self.logDir = logDir # near 5mb self.maxLogSize = 5000000 def startService(self): # logfile is a file-like object that supports rotation self.logFile = logfile.LogFile( self.logName, self.logDir, rotateLength=self.maxLogSize, maxRotatedFiles=50) #self.logFile.rotate() # force rotation each time restarted self.loclog = MyLog(self.logFile) self.loclog.start() def stopService(self): self.loclog.stop() self.logFile.close() del(self.logFile)
On Thu, Jan 21, 2010 at 7:02 PM, Maarten ter Huurne <maarten@treewalker.org> wrote:
Yes, may be. I start my daemon with /usr/bin/twistd -y mydaemon.py --logfile=/var/log/my.log --pidfile=/var/lock/mydaemon.pid How do I start twistd, so it doesn't produce own log file? Thanks for help! Pet
On 1/21/10 11:17 AM, Pet wrote:
You can customize the application to use your logfile and observer: http://twistedmatrix.com/documents/current/core/howto/application.html#auto6 e.g class MyLog(log.FileLogObserver): def emit(self, logEntryDict): log.FileLogObserver.timeFormat = '%Y-%m-%d %H:%M:%S' log.FileLogObserver.emit(self, logEntryDict) maxLogSize = 5000000 logFile = logfile.LogFile("my.log", "/var/log", rotateLength=maxLogSize, maxRotatedFiles=50) application = service.Application("myapp") application.setComponent(log.ILogObserver, MyLog(logFile).emit)
On Fri, Jan 22, 2010 at 12:13 AM, Lucas Taylor <ltaylor.volks@gmail.com> wrote:
Thanks for suggestion. I'll try it out as soon as I can. Currently I do it in that way: application = service.Application("MyService") myLogService = myLogService(LOG_NAME, LOG_DIR) myLogService.setServiceParent(application) what is the difference between creating service and setting setServiceParent and setComponent? Pet
On 1/22/10 3:15 AM, Pet wrote:
http://twistedmatrix.com/documents/current/core/howto/application.html The important aspect is the setComponent api. This is the part that lets you override the default logging behavior of twistd. It has nothing to do with the services that you register with the application using setServiceParent. If your MyLogService only does what you originally posted, you probably don't need all of that machinery. Using twistd will take care of starting and stopping logging for you. But, if you really want to use your service (say you want to force rotation on a restart), then you can do so. You just need to set the ILogObserver component on the application using your observer's emit function. e.g application = service.Application("MyService") myLogService = myLogService(LOG_NAME, LOG_DIR) myLogService.setServiceParent(application) application.setComponent(log.ILogObserver, myLogService.loclog.emit) Note that this won't work with your original MyLogService implementation without some reorganization (move logfile and loclog creation up to __init__)
On Fri, Jan 22, 2010 at 10:21 PM, Lucas Taylor <ltaylor.volks@gmail.com> wrote:
Hi, I'm getting an error while starting my service: Failed to load application: 'module' object has no attribute 'ILogObserver' I've imported log with from twisted.python import log Have no idea what is wrong... Pet
On Thu, Jan 21, 2010 at 7:02 PM, Maarten ter Huurne <maarten@treewalker.org> wrote:
Yes, may be. I start my daemon with /usr/bin/twistd -y mydaemon.py --logfile=/var/log/my.log --pidfile=/var/lock/mydaemon.pid How do I start twistd, so it doesn't produce own log file? Thanks for help! Pet
On 1/21/10 11:17 AM, Pet wrote:
You can customize the application to use your logfile and observer: http://twistedmatrix.com/documents/current/core/howto/application.html#auto6 e.g class MyLog(log.FileLogObserver): def emit(self, logEntryDict): log.FileLogObserver.timeFormat = '%Y-%m-%d %H:%M:%S' log.FileLogObserver.emit(self, logEntryDict) maxLogSize = 5000000 logFile = logfile.LogFile("my.log", "/var/log", rotateLength=maxLogSize, maxRotatedFiles=50) application = service.Application("myapp") application.setComponent(log.ILogObserver, MyLog(logFile).emit)
On Fri, Jan 22, 2010 at 12:13 AM, Lucas Taylor <ltaylor.volks@gmail.com> wrote:
Thanks for suggestion. I'll try it out as soon as I can. Currently I do it in that way: application = service.Application("MyService") myLogService = myLogService(LOG_NAME, LOG_DIR) myLogService.setServiceParent(application) what is the difference between creating service and setting setServiceParent and setComponent? Pet
On 1/22/10 3:15 AM, Pet wrote:
http://twistedmatrix.com/documents/current/core/howto/application.html The important aspect is the setComponent api. This is the part that lets you override the default logging behavior of twistd. It has nothing to do with the services that you register with the application using setServiceParent. If your MyLogService only does what you originally posted, you probably don't need all of that machinery. Using twistd will take care of starting and stopping logging for you. But, if you really want to use your service (say you want to force rotation on a restart), then you can do so. You just need to set the ILogObserver component on the application using your observer's emit function. e.g application = service.Application("MyService") myLogService = myLogService(LOG_NAME, LOG_DIR) myLogService.setServiceParent(application) application.setComponent(log.ILogObserver, myLogService.loclog.emit) Note that this won't work with your original MyLogService implementation without some reorganization (move logfile and loclog creation up to __init__)
On Fri, Jan 22, 2010 at 10:21 PM, Lucas Taylor <ltaylor.volks@gmail.com> wrote:
Hi, I'm getting an error while starting my service: Failed to load application: 'module' object has no attribute 'ILogObserver' I've imported log with from twisted.python import log Have no idea what is wrong... Pet
participants (5)
-
exarkun＠twistedmatrix.com -
Lucas Taylor -
Maarten ter Huurne -
Pet -
Ralph Meijer