Cropping log files

[Duncan Booth]

Kamus of Kadizhar <yan at NsOeSiPnAeMr.com> wrote in 
news:brmp3g$b4ar$1 at news3.infoave.net:

> The logging module lets me log events, but there aren't any tools for 
> managing the log files created in the way I need....  The 
> RotatingFileHandler rotates logs in the sense of logrotate, but what I 
> need is to keep only the last n records in one file.
> 

Keeping a fixed number of records in one file is difficult if the records 
aren't a fixed length, which is why most logging systems limit the number 
of records that are kept by the size of the data, rather than by the number 
of records.

If you can modify your requirements to say that you need to keep 'at least 
n records' then rotating file logger will do that, just make sure that the 
size that is kept in all files is at least n times the largest record you 
might log. Of course the records will then be split across multiple files.

If you really need to keep n records in one file, then the best thing is 
probably to write your own handler for the logging module and store the log 
records in a fixed length format in the file.

i.e. you choose a size larger than your largest log record and pad each log 
record to that length with nulls. Then you just write out to the file as 
though it were a circular buffer. You need some way of finding the first 
record again: one way is to always ensure you have at least one empty 
record in the file so when you first open the file you scan to the first 
empty record, then each time you write to the file you write the new record 
followed by a new empty record, then skip back so the empty record will be 
overwritten. When you reach record number n+1 in the file you simply wrap 
back to the start. You'll also want to write code to read a logfile 
starting after the last empty record.

If you can't set a hard upper bound on the size of a record you will need a 
more complicated file structure, or if that 'n' doesn't absolutely have to 
be set in stone simply allow occasional long records to wrap over into the 
next record and handle that case in both writing and reading code.

The NTEventHandler, (for those of us on Windows systems) does pretty much 
this last option, it uses event logs files which are a fixed length and 
wrap round, but individual records can span more than one block in the 
file, so the number of entries in a logfile is actually an upper limit.

-- 
Duncan Booth                                             duncan at rcp.co.uk
int month(char *p){return(124864/((p[0]+p[1]-p[2]&0x1f)+1)%12)["\5\x8\3"
"\6\7\xb\1\x9\xa\2\0\4"];} // Who said my code was obscure?