[Mailman-Users] Need help rebuilding archives...

[Phydeaux]

At 12:15 PM 3/26/2001 -0500, Phydeaux wrote:
>I have a list that has to have its archives rebuilt. The problem is that
>the first two years of archives do not exist in mbox format, as someone
>trimmed the mbox file way back when. This portion of the archives only
>exists in the regular text files, that the Mailman archiver created.
>
>I tried appending those two years of files together and changing the initial
>"From user at foo.bar" to "^N^NFrom user at foo.bar" to get Mailman to see
>the files as mbox format files. My local unix mail program (mailx) sees
>the file as having 2200 or so messages for these two years, but the
>archiver does not. It only sees two messages in total from it.

For those with any interest in this, the problem seems to be that
Mailman sees the break between messages a bit differently than
most mail packages. A simple "^N^NFrom " isn't enough to convince
Mailman that a new message has started. Instead it also looks for
the date/time on the "From" line. Inside Mailbox.py is this nifty chunk of
code that appears to cause this behaviour:

    _fromlinepattern = r'From \s*\S+\s+\w\w\w\s+\w\w\w\s+\d\d?\s+' \
                        r'\d\d?:\d\d(:\d\d)?(\s+\S+)?\s+\d\d\d\d\s*$'


I have no idea what the actual RFC for mbox format states, but for
now I have solved the mbox format part of the problem...

Now, when I get to the HTML generation of I get this wonderful set of messages
that indicates it has a problem with November 1999. I've looked at the
messages and can't find anything different about them. October's index (and
all the other files created) appears fine. When it gets to November 1999 I get
this:

Updating HTML for article 3265
Updating index files for archive [1999-November]
Traceback (innermost last):
   File "bin/arch", line 128, in ?
     main()
   File "bin/arch", line 118, in main
     archiver.close()
   File "/usr/local/mailman/Mailman/Archiver/HyperArch.py", line 911, in close
     self.update_dirty_archives()# Update all changed archives
   File "/usr/local/mailman/Mailman/Archiver/HyperArch.py", line 874, in 
update_dirty_archives
     self.update_archive(i)
   File "/usr/local/mailman/Mailman/Archiver/pipermail.py", line 303, in 
update_archive
     parameters=self.__set_parameters(archive)
   File "/usr/local/mailman/Mailman/Archiver/pipermail.py", line 224, in 
__set_parameters
     firstdate=self.database.firstdate(archive)
   File "/usr/local/mailman/Mailman/Archiver/HyperDatabase.py", line 188, 
in firstdate
     self.__openIndices(archive)
   File "/usr/local/mailman/Mailman/Archiver/HyperDatabase.py", line 237, 
in __openIndices
     self.__closeIndices()
   File "/usr/local/mailman/Mailman/Archiver/HyperDatabase.py", line 259, 
in __closeIndices
     index.close()
   File "/usr/local/mailman/Mailman/Archiver/HyperDatabase.py", line 167, 
in close
     fp.write(marshal.dumps(self.dict))
MemoryError

As always, any help/advice appreciated!

reb