[Moin-user] Rotating event-log file

Frank Lin PIAT fpiat at klabs.be
Sun May 10 08:26:11 EDT 2009


Hello,

I have written a script to rotate the event-log, preserving the previous
period's log entries.
Therefore, the statistics are never empty after log rotation.

It can also be used to rotate the edit-log every 3 months.

FYI, I have stressed it. It can "lose" 1 log line if you have more that
20 hits per seconds. (ie. about 70000 hits per hour !)

Comments are welcome.
[I know, it's NTRWTSTP: Not the right way to solve the problem. But I
want a solution for moin 1.7, not a patch for 1.9]

Franklin

#!/bin/sh
#
# rotate-and-keep.sh - Rotate a log file, keeping the last period.

#     This script is useful when you want to rotate a file 
#     periodically, but you want the "current" log file to always
#     contain two periods. (sometime needed for statistics)

#  Copyright 2009 Franklin Piat <URL:http://www.klabs.be/~fpiat/>
#
# This program is free software: you can redistribute it and/or modify
# it under the terms of the GNU General Public License as published by
# the Free Software Foundation, either version 2 of the License, or
# (at your option) any later version.
#
# This program is distributed in the hope that it will be useful,
# but WITHOUT ANY WARRANTY; without even the implied warranty of
# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
# GNU General Public License for more details.
#
# You should have received a copy of the GNU General Public License
# along with this program.  If not, see <http://www.gnu.org/licenses/>.

version=0.1

# LIMITATION
#
# Unique lines only
#    Use this script on logfiles that don't produce duplicate line
#    (i.e Each line MUST contain date and time, at least)
#
# Disk Space
#    The partition must have free space to make a copy of the last logfile
# 
#  ,--[ KNOWN BUG / DISCLAIMER ]------------------------------------------.
#  |                                                                      |
#  |  The traditional way to rotate a log file is to move it. Which is    |
#  |  guaranteed to be atomic and immediate.                              |
#  |                                                                      |
#  |  ** THIS SCRIPT ISN'T ATOMIC AND ISN'T IMMEDIATE **                  |
#  |  You may lose some new lines, appended while the rotation occurs.    |
#  |  (because it has to copy and manipulate the log file. Typically if   |
#  |  you have more than 10 lines of log per seconds... that is > 100000  |
#  |  page hit per day!)                                                  |
#  `----------------------------------------------------------------------'


LOGFILE=event-log
ARCHIVE=event-log.$(date +%F_%T)
TMPFILE=event-log.tmp
LASTLINE=lastline


set -e
pivotline=0
if [ -s "$LASTLINE" -a -s "$LOGFILE" ]; then
	pivotline=$(grep  -s -n -F -f $LASTLINE $LOGFILE | tail -n 1 | cut -d ":" -f 1)
	#DEBUG: printf "Rotate line #$pivotline: " >&2 ; cat "$LASTLINE" > &2
fi

if [ "$pivotline" -gt 1 ] ; then
	pivotline=$(( $pivotline + 1 ))
	mv "$LOGFILE" "$TMPFILE"

	if sed -n $pivotline,\$p "$TMPFILE" > "$ARCHIVE" ; then
		cp "$ARCHIVE" "$TMPFILE" 
		## TEST: for test purpose, you can insert a sleep here to
		##       simulate the copy of large log file.
		#sleep 1

		# Try to catch new lines (that occured during the "cp" above)
		if [ -s "$LOGFILE" ]; then
			 cat "$LOGFILE" 2>/dev/null | tee -a "$ARCHIVE" >> "$TMPFILE"
		fi

		mv "$TMPFILE" "$LOGFILE"
		tail -n 1 "$ARCHIVE" > "$LASTLINE"
	else
		#Restore the logfile in case of error (especially disk-space!)
		echo "LOG ROTATION ABORTED: probably insufficient disk-space." >&2
		mv "$TMPFILE" "$LOGFILE"
		exit 1
	fi
else
	if [ -s "$LOGFILE" ]; then
		cp "$LOGFILE" "$ARCHIVE"
		tail -n 1 "$ARCHIVE" > "$LASTLINE"
	else
		cp /dev/null "$LASTLINE"
	fi
fi
#
# To test/stress this script you can generate a fake log using
# while true; do date +%s.%N | tee -a event-log >> log; sleep 0.01 ; done





More information about the Moin-user mailing list