[Mailman-Users] moving archives before migration

[Mark Sapiro]

On 05/24/2013 11:10 AM, Jason Fayre wrote:
> Hello,
> I am in the process of migrating data from a server to a new vps. The
> server is hosting around 200 lists.  The mail archives on this server
> are around 30GB.


You don't need to move the entire archive. You probably need to rebuild
the archives on the new server anyway to fix things like links to
scrubbed attachments, so with one important exception, all you need are
the archives/private/LISTNAME.mbox/LISTNAME.mbox files which are used to
rebuild the archives on the new server with 'bin/arch --wipe'.

The exception is if any lists have scrub_nondigest = Yes, any
attachments were scrubbed during incoming message processing and are not
in the LISTNAME.mbox file. Thus, for those lists if any the
archives/private/LISTNAME/attachments directory needs to be moved and
some process run to fix the links in archived messages.


> The only way I can get the data to the new server is over the internet. 
> Due to the speed of the DSL link (700k upload), I need to turn off the
> mail server in order to migrate data at any sort of reasonable speed.
> What I would like to do is move the list archives out of the mailman
> directory temporarily while we migrate everything else over.  Then, once
> we have the new server set up, move the archives.


I think this could work and as I say above, at most you only need to
ultimately move the LISTNAME.mbox files and maybe some attachments/
directories.


> If this makes any sort of sense, can someone point me in the right
> direction on how we might accomplish this.  If we can't do the archives
> later, I'm going to need to have the mail server down for about a week
> while the data moves over.  I'm trying to avoid this, since some of our
> lists are fairly high traffic.


Presumably, you're OK with not having archives for the time the move takes.

Here's what I'd do.

1) Put ARCHIVE_TO_MBOX = 1 in mm_cfg.py on the old server and restart
Mailman and don't forget to remove it on the new server. This will turn
off pipermail archiving and only update the LISTNAME.mbox files.
2) Move aside or just delete the archives/private/LISTNAME/ directories
except maybe the archives/private/LISTNAME/attachments/ directories for
any scrub_nondigest = Yes lists.
3) After the lists are running on the new server, move the LISTNAME.mbox
files to some place else on the new server. You don't want to overwrite
the new LISTNAME.mbox files.
4) If there are any scrub_nondigest lists, they should be OK because
their attachments/ directories got moved with the rest of the list stuff.
5) Stop Mailman on the new server and merge the old and new
LISTNAME.mbox files and run bin/arch --wipe for each list.
6) Restart Mailman

If steps 3 - 5 don't take too much time because of not moving the
pipermail archives, you could just move the LISTNAME.mbox files into
place and run the bin/arch --wipe before ever starting Mailman on the
new server.

-- 
Mark Sapiro <mark at msapiro.net>        The highway is for gamblers,
San Francisco Bay Area, California    better use your sense - B. Dylan