Just noticed a minor bug in qrunner. when qrunner starts up, it opens the log files, and leaves them open.
this is fine -- except when the logs get rolled and restarted. When you move the logs off and gzip them for storage, qrunner continues writing to the (now unlinked) old files, not the new, zeroed ones. So all of the qrunner stat and errror and whatever data from the time you roll the logs until that instance of qrunner exits is gone (which, if you've modified qrunner lifetime values, can become significant on a busy system)
Since mailman isn't oding much log processing, not a big deal. but for sites that are (for instance) charging for list usage, or sites that have implemented reporting, there are potential holes in the data, and if they're busy so they have extended qrunner times, significant holes.
while open/write/close is expensive, given all of the other stuff qrunner's doing and the relative lack of logging, I don't think it's significant. Alternatively, it can notice the files have been moved/zeroed, or perhaps accept a HUP like syslog...
-- Chuq Von Rospach, Internet Gnome http://www.chuqui.com [chuqui@plaidworks.com = me@chuqui.com = chuq@apple.com] Yes, yes, I've finally finished my home page. Lucky you.
I'm out of my mind, but feel free to leave a message...
on 5/31/01 12:23 AM, "Chuq Von Rospach" chuqui@plaidworks.com wrote:
Just noticed a minor bug in qrunner. when qrunner starts up, it opens the log files, and leaves them open.
this is fine -- except when the logs get rolled and restarted. When you move the logs off and gzip them for storage, qrunner continues writing to the (now unlinked) old files, not the new, zeroed ones. So all of the qrunner stat and errror and whatever data from the time you roll the logs until that instance of qrunner exits is gone (which, if you've modified qrunner lifetime values, can become significant on a busy system)
Since mailman isn't oding much log processing, not a big deal. but for sites that are (for instance) charging for list usage, or sites that have implemented reporting, there are potential holes in the data, and if they're busy so they have extended qrunner times, significant holes.
while open/write/close is expensive, given all of the other stuff qrunner's doing and the relative lack of logging, I don't think it's significant. Alternatively, it can notice the files have been moved/zeroed, or perhaps accept a HUP like syslog...
The files should remain open... Because, exactly as you suggest, there is a sever penalty for open/write/close.
The way that Apache HTTPd works (as well as many other daemons) is that you move the log files out of the way and then you HUP Apache which will re-create the log files. Accepting HUP's (or a graceful restart) is functionality that should be added to qrunner if it isn't there already.
-jon
-- If you come from a Perl or PHP background, JSP is a way to take your pain to new levels. --Anonymous http://jakarta.apache.org/velocity/ymtd/ymtd.html
"CVR" == Chuq Von Rospach chuqui@plaidworks.com writes:
CVR> Just noticed a minor bug in qrunner. when qrunner starts up,
CVR> it opens the log files, and leaves them open.
CVR> while open/write/close is expensive, given all of the other
CVR> stuff qrunner's doing and the relative lack of logging, I
CVR> don't think it's significant. Alternatively, it can notice
CVR> the files have been moved/zeroed, or perhaps accept a HUP
CVR> like syslog..."JS" == Jon Stevens jon@latchkey.com writes:
JS> The way that Apache HTTPd works (as well as many other
JS> daemons) is that you move the log files out of the way and
JS> then you HUP Apache which will re-create the log
JS> files. Accepting HUP's (or a graceful restart) is
JS> functionality that should be added to qrunner if it isn't
JS> there already.This turned out to fairly easy to add, so I have. :)
SIGHUP on the qrunner master process now closes all the log files. They will be reopened on the next write.
-Barry
On Friday, June 1, 2001, at 10:31 PM, Barry A. Warsaw wrote:
This turned out to fairly easy to add, so I have. :)
SIGHUP on the qrunner master process now closes all the log files. They will be reopened on the next write.
by the way, another app susceptible to this is mailpasswd.
It'd be nice if there were some way to tell these tools to shut down conveniently, and if possible, snapshot and restart where they left off. But that's probably asking for gold leaf instead of gold paint...
-- Chuq Von Rospach, Internet Gnome http://www.chuqui.com [chuqui@plaidworks.com = me@chuqui.com = chuq@apple.com] Yes, yes, I've finally finished my home page. Lucky you.
Stress is when you wake up screaming and you realize you haven't fallen asleep yet.
participants (3)
-
barry＠digicool.com -
Chuq Von Rospach -
Jon Stevens