[Mailman-Users] Trimming archives

[Brad Knowles]

At 9:10 AM -0500 2005-10-31, Darren G Pifer wrote:

>         In reviewing posts to this mailing list, I found that it is not
>  possible to prune the archives.

	Not automatically, no.

>                                   However, we will need to do something
>  shortly because the filesystem that we have mailman on is now 84% full,
>  mostly due to the growing archives. We are using LVM under Linux and I
>  can increase the space but I would like to see if we can compress any
>  older files in the archive directory first.

	There are cron jobs for Mailman that are detailed in the 
documentation which will periodically compress the old web-accessible 
text format archives, yes.  The HTML-format web archives themselves 
cannot be compressed, however.  And the original 7th edition 
mbox-format mailbox (from which all the public archives are derived) 
cannot be compressed as-is.

	So, there's a limit to the amount of space that you would be able 
to recover.  Moreover, your choices are limited in terms of the 
compression algorithms you can choose, at least without making any 
modifications to the Mailman source code -- for the standard cron 
jobs, gzip is used and not bzip2.

>                                               For instance, we have one
>  mbox, ./archives/private/opr.mbox/opr.mbox that is 700MB is growing
>  everyday. It is our operators list. Can I gzip it and create a new
>  opr.mbox by touching it?

	This is the unprocessed "source" archive mailbox, from which all 
public archives are generated.  This is a record of exactly what came 
in and in what order.

	I would be inclined to rename it first, using an extension based 
on the date when the archive was to be compressed, then create the 
new one with the same ownership and permissions.  You could then 
compress the old file at your leisure.

	Of course, if you ever have to rebuild your public archives, 
you'll need to make sure that you process the messages in the oldest 
compressed archive first, otherwise all the message numbers will get 
out-of-whack.

>                            Also, there are some text files
>  ./archives/private/opr/2005-October.txt that is 500+ MB.

	Those are the processed text-format web-accessible archives which 
were created based on the source archive mailbox contents.  These are 
the archives which are typically compressed in the standard cron jobs.

>                                                            Can .txt files
>  compressed and still useable by the system?

	Mailman doesn't make any attempt to use these files.  These files 
are produced by Mailman for use by humans.  Once they've been 
compressed, it's fine to leave them in compressed format, since they 
would only be downloaded by people accessing the archive system and 
who wish to see text-format archives as opposed to the HTML-format 
archives.

>                                              Is anybody doing anything to
>  archive older, unused mailman archives to tape? We use Tivoli (TSM) here
>  and wanted to know if anybody has something automated for this.

	There are no standard automated procedures I know of to split and 
compress old 7th edition mbox-format source archives.  They are 
always assumed to be in pure 7th edition mbox text format, and if you 
want to split and compress them you will need to come up with your 
own procedures for doing that.

-- 
Brad Knowles, <brad at stop.mail-abuse.org>

"Those who would give up essential Liberty, to purchase a little
temporary Safety, deserve neither Liberty nor Safety."

     -- Benjamin Franklin (1706-1790), reply of the Pennsylvania
     Assembly to the Governor, November 11, 1755

   SAGE member since 1995.  See <http://www.sage.org/> for more info.