[Mailman-Users] Problems regenerating archives from mbox files

[Mark Sapiro]

Hank van Cleef wrote:
>
>Thus far, it appears that we've gotten a complete build, but have
>problems with year 2002.  About 250 posts have their headers archived
>properly, but without the message text.  That text is archived as 
>"no subject" on the date arch was run.  I've done a quick check, just
>to idenfity the problem period, and so see what's in the mbox files
>for the affected posts.


This is a symptom of unescaped From_ lines in the bodies of the
messages. Run bin/cleanarch against this .mbox


>Qhuestion number one, of course, is why is arch behaving this way?


See above


>This is affectingn one year's mbox file's builds.  That mbox (year
>2002) was generated under Mailman, and I think it was an early 2.x rev
>that was replaced over New Year's 2003.  
>
>Question: Does the --wipe option delete all the archives, or does it
>see what period the selected mbox covers?  I've assumed it's all
>archives, so doing an incremental rebuild will require storing
>anything previously build elsewhere?


--wipe removes the entire archive. To do an incremental build, you
specify --wipe on only the first bin/arch command.


>Question: Is there a way to do an incremental rebuild and have it
>replace already-built archives for a given time period?  Or is that a
>manual rm job?


It's a manual rm, but easy enough unless there are some to remove and
some to keep in the same volume (month).


>Other things being equal, my plan of attack right now is to set up 
>an empty archive directory, do incremental builds in it, then move
>each increment to a backup directory, clean out the build directory,
>and do the next increment.  When all increments are built clean, I
>can then move the backup to production.  If there's a better way,
>I'm all ears.


This is a bad idea. It will work, but if you have messages for the same
volume in two increments, there will be a conflict.

Just do bin/arch --wipe with the first increment and then bin/arch
without the --wipe for the succeeding increments. After each
increment, check and if the archive is OK, copy the entire archive to
the backup. If it's not OK, restore the previous backup and fix what's
wrong with that increment's .mbox.

-- 
Mark Sapiro <mark at msapiro.net>        The highway is for gamblers,
San Francisco Bay Area, California    better use your sense - B. Dylan