[ mailman-Patches-1123383 ] Daily Status Report script...
SourceForge.net
noreply at sourceforge.net
Thu Jan 26 02:55:05 CET 2006
Patches item #1123383, was opened at 2005-02-15 20:14
Message generated for change (Comment added) made by shub
You can respond by visiting:
https://sourceforge.net/tracker/?func=detail&atid=300103&aid=1123383&group_id=103
Please note that this message will contain a full copy of the comment thread,
including the initial issue submission, for this request,
not just the latest update.
Category: list administration
Group: Mailman 2.1
Status: Open
Resolution: None
Priority: 5
Submitted By: Brad Knowles (shub)
Assigned to: Nobody/Anonymous (nobody)
Summary: Daily Status Report script...
Initial Comment:
Folks,
I quickly whacked together a Daily Status Report script for
Mailman (using Bourne shell, not Python ;), and thought that other
folks might be interested in seeing it.
The basic concept is a program that gets fired off at 23:59 every
night, and goes through a variety of log files looking for entries
specific to that date, and indicating problems or certain types of
activity that might be of interest to someone trying to administer
the server. It also does an "ls -la" of /usr/local/mailman/qfiles/*,
so that you can see what is in the queue at the time of the running
of the script.
My concept was that this daily report would get e-mailed to the
admin, or posted to a "reports" mailing list, where they could be
archived and kept for future reference.
The script does not (yet) do any statistics calculations, although it
should be relatively easy to hack together some basic stats using
awk, sort, etc....
Anyway, I thought I'd share it and let folks take a look at it, and if
anyone has any recommended improvements, we can incorporate
those and share them back out with everyone.
The code is written under a BSD-style license, so if you don't want
to contribute any changes back to me, that's okay. Of course, I
would prefer that you did, but I leave the choice up to you.
----------------------------------------------------------------------
>Comment By: Brad Knowles (shub)
Date: 2006-01-26 01:55
Message:
Logged In: YES
user_id=18417
Folks,
Okay, I just uploaded version 0.0.15, which addresses the most recent
comments here from Mark and Tom, and makes a few additional tweaks as
well. See the version comments at the top of the script.
----------------------------------------------------------------------
Comment By: Mark Sapiro (msapiro)
Date: 2005-11-06 18:25
Message:
Logged In: YES
user_id=1123998
The vette log summary lists posts held for moderation
individually under "Other Errors:". The following patch
(watch out for wrapped lines) summarizes them by list instead.
--- mmdsr 2005-09-29 21:31:33.000000000 -0700
+++ mmdsrx 2005-11-06 08:20:58.835895418 -0800
@@ -415,10 +415,15 @@
echo "------------------------------" >> $TMP
$GREP -i 'Posting to a moderated newsgroup' $TMPLOG |
$AWK '{ print $6 }' | $SORT | $UNIQ -c | $SORT -nr >> $TMP
+ echo "" >> $TMP
+ echo "Post to moderated list (by list):" >> $TMP
+ echo "------------------------------" >> $TMP
+ $GREP -i 'Post to moderated list' $TMPLOG |
$AWK '{ print $6 }' | $SORT | $UNIQ -c | $SORT -nr >> $TMP
+
echo "" >> $TMP
echo "Other Errors:" >> $TMP
echo "------------------------------" >> $TMP
- $EGREP -vi '(Post by non-member|suspicious
header|message approved|Discarded posting|bulk message
discarded|junk message discarded|Message has implicit
destination|Posting to a moderated newsgroup|Message
discarded, msgid)' $TMPLOG | $SED 's/^.* ([0-9]*) //' |
$SORT | $UNIQ -c | $SORT -nr >> $TMP
+ $EGREP -vi '(Post by non-member|suspicious
header|message approved|Discarded posting|bulk message
discarded|junk message discarded|Message has implicit
destination|Posting to a moderated newsgroup|Post to
moderated list|Message discarded, msgid)' $TMPLOG | $SED
's/^.* ([0-9]*) //' | $SORT | $UNIQ -c | $SORT -nr >> $TMP
else
----------------------------------------------------------------------
Comment By: Tom G. Christensen (tgc99)
Date: 2005-10-19 08:23
Message:
Logged In: YES
user_id=1159458
ps output on solaris is full of whitespace but a further
echo get's rid of it.
The lines in the smtp log are sometimes broken up by a
newline (right before the msgid) which throws of the
summary. Piping it through sed first will rejoin the broken
lines.
Use $AWK instead of awk.
Patch inserted below:
--- mmdsr.orig 2005-10-19 09:42:30.000000000 +0200
+++ mmdsr 2005-10-19 09:44:23.000000000 +0200
@@ -203,7 +203,8 @@
# there is an easier cross-platform way to do it,
please let me know.
###############################################################################
-MYUID=`$PS -o user -p $$ | $TAIL -1`
+GRABUID=`$PS -o user -p $$ | $TAIL -1`
+MYUID=`echo $GRABUID`
RUNAS="mailman"
###############################################################################
@@ -254,7 +255,7 @@
$TOUCH $TMPLOG
echo "Log file: $LOG" >> $TMP
echo "==============================" >> $TMP
- $GREP -si "^$DAY [0-9][0-9:]* $YEAR" $LOGDIR/$LOG >>
$TMPLOG
+ $SED -e :a -e '$!N;s/\n //;ta' -e 'P;D' $LOGDIR/$LOG |
$GREP -si "^$DAY [0-9][0-9:]* $YEAR" >> $TMPLOG
if [ -f "$LOGDIR/${LOG}" ] ; then
@@ -264,7 +265,7 @@
echo "Hourly Summary of Posts" >> $TMP
echo "-----------------------" >> $TMP
- $SED -e 's/^[A-Z][a-z][a-z] *[0-9]* //' -e
's/:.*$//' $TMPLOG | $UNIQ -c | $SORT -n +1 | awk '{ printf(
"%8d %02d:00-%02d
:59\n", $1, $2, $2 ) }' >> $TMP
+ $SED -e 's/^[A-Z][a-z][a-z] *[0-9]* //' -e
's/:.*$//' $TMPLOG | $UNIQ -c | $SORT -n +1 | $AWK '{
printf( "%8d %02d:00-%02
d:59\n", $1, $2, $2 ) }' >> $TMP
echo "" >> $TMP
echo "Post Count by List" >> $TMP
@@ -295,7 +296,7 @@
echo "" >> $TMP
echo "Hourly Summary of Messages Sent" >> $TMP
echo "-------------------------------" >> $TMP
- $SED -e 's/^[A-Z][a-z][a-z] *[0-9]* //' -e
's/:.* for / /' -e 's/ recips,.*$//' $TMPLOG | awk '{
val=int($1); sum[val]+=$2
} END { for (i=0; i<24; i++) { printf "%8d
%02d:00-%02d:59\n", sum[i], i, i } }' >> $TMP
+ $SED -e 's/^[A-Z][a-z][a-z] *[0-9]* //' -e
's/:.* for / /' -e 's/ recips,.*$//' $TMPLOG | $AWK '{
val=int($1); sum[val]+=$
2 } END { for (i=0; i<24; i++) { printf "%8d
%02d:00-%02d:59\n", sum[i], i, i } }' >> $TMP
else
----------------------------------------------------------------------
Comment By: Brad Knowles (shub)
Date: 2005-09-22 23:17
Message:
Logged In: YES
user_id=18417
Okay, I took Mark's comments and incorporated them. Adrian Wells also
pointed out a log file difference between Mailman 2.1.5 and 2.1.6 that
caused the summary of the "smtp" log to be munged.
I've now fixed these bugs, deleted the old file, and uploaded the new one
(version 0.0.12).
Thanks!
----------------------------------------------------------------------
Comment By: Mark Sapiro (msapiro)
Date: 2005-09-11 02:22
Message:
Logged In: YES
user_id=1123998
Oooops! I mistakenly thought I could add the mmdsr.patch as
a downloadable file, but I can't so here it is. Watch out
for wrapped lines ...
--- mmdsr.orig 2005-09-06 20:37:53.000000000 -0700
+++ mmdsr 2005-09-10 18:33:14.532393572 -0700
@@ -132,6 +132,7 @@
MMDIR="/usr/local/mailman"
TMPDIR="/tmp"
+LOGDIR="/var/log/mailman"
###############################################################################
# Maximum number of subdirectory entries to display in report
@@ -234,9 +235,9 @@
$TOUCH $TMPLOG
echo "Log file: $LOG" >> $TMP
echo "==============================" >> $TMP
- $GREP -si "^$DAY [0-9][0-9:]* $YEAR" logs/$LOG >> $TMPLOG
+ $GREP -si "^$DAY [0-9][0-9:]* $YEAR" $LOGDIR/$LOG >>
$TMPLOG
- if [ -f "logs/${LOG}" ] ; then
+ if [ -f "$LOGDIR/${LOG}" ] ; then
if [ "${LOG}" = "post" ] ; then
@@ -304,9 +305,9 @@
$TOUCH $TMPLOG
echo "Log file: $LOG" >> $TMP
echo "==============================" >> $TMP
- $GREP -si "^$DAY [0-9][0-9:]* $YEAR" logs/$LOG >> $TMPLOG
+ $GREP -si "^$DAY [0-9][0-9:]* $YEAR" $LOGDIR/$LOG >>
$TMPLOG
- if [ -f "logs/${LOG}" ] ; then
+ if [ -f "$LOGDIR/${LOG}" ] ; then
if [ "${LOG}" = "error" ] ; then
@@ -453,4 +454,4 @@
$CAT $TMP
fi
-$RM $TMP
+$RM -f $TMP $TMPLOG
----------------------------------------------------------------------
Comment By: Mark Sapiro (msapiro)
Date: 2005-09-11 02:16
Message:
Logged In: YES
user_id=1123998
I tried the script and other than the expected site specific
changes, I found two issues which are both addressed by the
patch in the mmdsr.patch file.
1) Mailman's log files are not necessarily in
$var_prefix/logs/ - they can be in any directory. The patch
adds a LOGDIR directory independant of MMDIR.
2) The $TMPLOG file is not removed at completion. The patch
removes it.
----------------------------------------------------------------------
Comment By: Brad Knowles (shub)
Date: 2005-09-06 14:06
Message:
Logged In: YES
user_id=18417
Okay, I've deleted the old version of the mmdsr script that was attached,
and uploaded the latest version. This is what we're currently using to
monitor the lists on python.org, and we have found it very useful.
Any comments you may have will be appreciated.
----------------------------------------------------------------------
Comment By: Brad Knowles (shub)
Date: 2005-02-22 20:10
Message:
Logged In: YES
user_id=18417
The UID variable in the current code was already replaced by MYUID,
because I got complaints on other platforms. But UID wasn't available to
me as a useful constant, so I had to use something else to obtain the
value.
The recommended patch from tgc99 does work, and I will be uploading a
new version of the code soon.
----------------------------------------------------------------------
Comment By: adrianwi (adrianwi)
Date: 2005-02-22 15:22
Message:
Logged In: YES
user_id=1175103
Use of variable named UID does work well with OS X (version
10.2.8). Apparently the variable UID is a constant already
in use. When trying to the run the script without
modification, I was receiving the following error message:
UID: readonly variable
This issue was resolved by changing the name of variable,
UID, to something else, such as MMUID. Works fine with this
change.
As an aside (& for what it is worth), the UID grab command
suggested by tgc99 on 2005-02-16 03:15 works on this system
(OS X - version 10.2.8)
----------------------------------------------------------------------
Comment By: Tom G. Christensen (tgc99)
Date: 2005-02-16 08:15
Message:
Logged In: YES
user_id=1159458
The current UID grab command doesn't work on Solaris (2.6 &
8 tested).
I'd recommend this instead:
ps -o user -p $$|tail -1
This is tested and works on RH 6.2, RH 7.3, RHEL 2.1, RHEL3,
FC3, FreeBSD 4.9, Solaris 2.6, 8.
----------------------------------------------------------------------
You can respond by visiting:
https://sourceforge.net/tracker/?func=detail&atid=300103&aid=1123383&group_id=103
More information about the Mailman-coders
mailing list