Howdy. I am currently running 2.1b5 of Mailman and am trying to sort out an issue with archiving that has crept up.
The problem has been mentioned previously from what I can tell but no resolution seems to have been mentioned.
What the problem is that list archives (for reasons I won't bore you with) have a number of SPAM message in them with all sorts of random encoding types and other mangled garbage. What happens is that when the archiver gets to the point of writing the archive, the encoding type test generates an error and the whole archiving process grinds to a crashing halt. These are busy lists and the mbox archive takes a very long time to parse and there just is not enough time in the day to search for the offending message, chop it out and wait another 45 minutes or more until the archives are regenerated to hit the next garbled header, etc. This will also continue to be a problem if any future SPAM messages sneak in via forged headers, etc.
The issue appears to be with:
/usr/local/mailman/Mailman/Archiver/HyperArch.py
Traceback (most recent call last): File "./bin/arch", line 173, in ? main() File "./bin/arch", line 163, in main archiver.close() File "/usr/local/mailman/Mailman/Archiver/pipermail.py", line 303, in close self.update_dirty_archives() File "/usr/local/mailman/Mailman/Archiver/pipermail.py", line 517, in update_dirty_archives self.update_archive(i) File "/usr/local/mailman/Mailman/Archiver/HyperArch.py", line 1058, in update_archive self.__super_update_archive(archive) File "/usr/local/mailman/Mailman/Archiver/pipermail.py", line 423, in update_archive self._update_simple_index(hdr, archive, arcdir) File "/usr/local/mailman/Mailman/Archiver/pipermail.py", line 444, in _update_simple_index self.write_index_entry(article) File "/usr/local/mailman/Mailman/Archiver/HyperArch.py", line 980, in write_index_entry subject = self.get_header("subject", article) File "/usr/local/mailman/Mailman/Archiver/HyperArch.py", line 1007, in get_header return unicode(result, article.charset) TypeError: unicode() argument 2 must be string, not None
What I want is the archiver to default to english if it cannot figure out the encoding so that at least the archiver will not die.
So two questions:
What is a valid encoding type to pass as default to the unicode call?
Secondly, is there any danger in changing the fallback option to always use a
specific charset? I'd rather have gibberish than a process that dies.
Basically, around line 1007 in "/usr/local/mailman/Mailman/Archiver/HyperArch.py" I want to change:
if isinstance(result, types.UnicodeType): return result try: return unicode(result, article.charset)
to
if isinstance(result, types.UnicodeType): return result try: return unicode(result, "some string") # never fail!
Thanks for any suggestions.
Cheers