A few weeks back (http://mail.python.org/pipermail/mailman-developers/ 2002-November/013960.html), Andrew Clark reported a traceback from bin/arch on some lists some of the time. I've just hit the same bug (our tracebacks are identical):
$ ./bin/arch <listname> [...churn, churn, churn...] #04077 <mailman.4063.1039018365.21495.mems-talk@memsnet.org> Updating index files for archive [2001-July] Date Subject Author Thread Computing threaded index Updating HTML for article 8892 Updating HTML for article 8893 Updating HTML for article 8894 Updating HTML for article 8895 Updating HTML for article 8896 Updating HTML for article 8897 Updating HTML for article 8898 Updating HTML for article 8899 Updating HTML for article 8891 Updating HTML for article 8900 Updating HTML for article 8901 Updating HTML for article 8902 Updating HTML for article 8903 Updating HTML for article 8904 Updating HTML for article 5007 Traceback (most recent call last): File "./bin/arch", line 173, in ? main() File "./bin/arch", line 163, in main archiver.close() File "/usr/local/mailman/Mailman/Archiver/pipermail.py", line 303, in close self.update_dirty_archives() File "/usr/local/mailman/Mailman/Archiver/pipermail.py", line 517, in update_d irty_archives self.update_archive(i) File "/usr/local/mailman/Mailman/Archiver/HyperArch.py", line 1058, in update_ archive self.__super_update_archive(archive) File "/usr/local/mailman/Mailman/Archiver/pipermail.py", line 425, in update_a rchive self._update_thread_index(archive, arcdir) File "/usr/local/mailman/Mailman/Archiver/pipermail.py", line 479, in _update_ thread_index self.update_article(arcdir, article, a1, a3) File "/usr/local/mailman/Mailman/Archiver/HyperArch.py", line 1223, in update_ article f.write(article.as_html()) File "/usr/local/mailman/Mailman/Archiver/HyperArch.py", line 407, in as_html d['listurl'] = self._mlist.GetScriptURL('listinfo', absolute=1) AttributeError: Article instance has no attribute '_mlist'
Andrew said that renaming away the existing HTML archive did the trick -- I'll do that, but I don't like making the archives for this list unavailable even for a few minutes. (It takes a while to generate the whole archive, which goes back to 1994.) Or should I "cvs up" instead, ie. has this bug been fixed?
Thanks --
Greg-- Greg Ward - software developer gward@mems-exchange.org MEMS Exchange http://www.mems-exchange.org
"GW" == Greg Ward <gward@mems-exchange.org> writes:
GW> Andrew said that renaming away the existing HTML archive did
GW> the trick -- I'll do that, but I don't like making the
GW> archives for this list unavailable even for a few minutes.
GW> (It takes a while to generate the whole archive, which goes
GW> back to 1994.) Or should I "cvs up" instead, ie. has this bug
GW> been fixed?Greg, I believe this bug is fixed in cvs, so I'd recommend cvs up'ing. However, if any new messages arrived while you were running 2.1b4 then you should also run b4b5-archfix after up'ing.
If, after all that you still get this exception, I wanna know about it.
-Barry
On 04 December 2002, Barry A. Warsaw said:
Greg, I believe this bug is fixed in cvs, so I'd recommend cvs up'ing. However, if any new messages arrived while you were running 2.1b4 then you should also run b4b5-archfix after up'ing.
The list in question never ran under 2.1b4 -- I got nervous about all the problems people were having with the archiver, so didn't upgrade it until 2.1b5 was out. And, just for the heck of it, I did re-run b4b5-archfix -- no point though, since the grep for '_mlist' turned up nothing.
If, after all that you still get this exception, I wanna know about it.
Yes, I'm still getting it with the current CVS code. It comes in exactly the same place in the execution of bin/arch, and the traceback is basically identical -- mainly line number differences, and your recent patch to Mailman/Archiver/HyperArch.py.
Here's are arch's death throes:
[...] Updating HTML for article 8903 Updating HTML for article 8904 Updating HTML for article 5007 Traceback (most recent call last): File "./bin/arch", line 186, in ? main() File "./bin/arch", line 176, in main archiver.close() File "/usr/local/mailman/Mailman/Archiver/pipermail.py", line 303, in close self.update_dirty_archives() File "/usr/local/mailman/Mailman/Archiver/pipermail.py", line 517, in update_dirty_archives self.update_archive(i) File "/usr/local/mailman/Mailman/Archiver/HyperArch.py", line 1072, in update_archive self.__super_update_archive(archive) File "/usr/local/mailman/Mailman/Archiver/pipermail.py", line 425, in update_archive self._update_thread_index(archive, arcdir) File "/usr/local/mailman/Mailman/Archiver/pipermail.py", line 479, in _update_thread_index self.update_article(arcdir, article, a1, a3) File "/usr/local/mailman/Mailman/Archiver/HyperArch.py", line 1247, in update_article f.write(article.as_html()) File "/usr/local/mailman/Mailman/Archiver/HyperArch.py", line 411, in as_html emailurl = self._mlist.GetListEmail() AttributeError: Article instance has no attribute '_mlist'
-- Greg Ward - software developer gward@mems-exchange.org MEMS Exchange http://www.mems-exchange.org
On 04 December 2002, I said:
Yes, I'm still getting it with the current CVS code. It comes in exactly the same place in the execution of bin/arch, and the traceback is basically identical -- mainly line number differences, and your recent patch to Mailman/Archiver/HyperArch.py.
Found the problem and uploaded a patch to my bug report. Article.setListIfUnset() was, umm, completely broken.
In related news, bin/arch bloated up to 200 MB of RAM trying to rebuild this archive before I killed it. Ouch! I thought that had been fixed too.
Greg"GW" == Greg Ward <gward@mems-exchange.org> writes:
GW> In related news, bin/arch bloated up to 200 MB of RAM trying
GW> to rebuild this archive before I killed it. Ouch! I thought
GW> that had been fixed too.That's not surprising, since pipermail keeps everything in memory as it's building the archive. Which is why bin/arch has the --start and --end options, so you can build an archive in chunks.
-Barry
On 04 December 2002, I said:
Andrew said that renaming away the existing HTML archive did the trick -- I'll do that, but I don't like making the archives for this list unavailable even for a few minutes.
Arrghh! I did this and now bin/arch goes a lot farther, but it still crashes eventually:
[...] Updating HTML for article 4073 Updating HTML for article 4074 Updating HTML for article 4076 Pickling archive state into /usr/local/mailman/archives/private/mems-talk/pipermail.pck Traceback (most recent call last): File "./bin/arch", line 173, in ? main() File "./bin/arch", line 161, in main archiver.processUnixMailbox(fp, start, end) File "/usr/local/mailman/Mailman/Archiver/pipermail.py", line 557, in processUnixMailbox a = self._makeArticle(m, self.sequence) File "/usr/local/mailman/Mailman/Archiver/HyperArch.py", line 619, in _makeArticle mlist=self.maillist) File "/usr/local/mailman/Mailman/Archiver/HyperArch.py", line 261, in __init__ self.check_header_charsets(charset) File "/usr/local/mailman/Mailman/Archiver/HyperArch.py", line 350, in check_header_charsets author, a_charset = self.decode_charset(self.author) File "/usr/local/mailman/Mailman/Archiver/HyperArch.py", line 379, in decode_charset h = make_header(pairs, 99999) File "/usr/local/mailman/pythonlib/email/Header.py", line 123, in make_header h.append(s, charset) File "/usr/local/mailman/pythonlib/email/Header.py", line 230, in append ustr = unicode(s, incodec) LookupError: unknown encoding
I presume this is some Asian encoding that's not *supposed* to appear in an English-language list, but does. Grumble. I'll see if I can figure out the offending message.
Greg-- Greg Ward - software developer gward@mems-exchange.org MEMS Exchange http://www.mems-exchange.org
On 04 December 2002, I said:
[...] Updating HTML for article 4073 Updating HTML for article 4074 Updating HTML for article 4076 Pickling archive state into /usr/local/mailman/archives/private/mems-talk/pipermail.pck Traceback (most recent call last): File "./bin/arch", line 173, in ? main() [...] File "/usr/local/mailman/pythonlib/email/Header.py", line 230, in append ustr = unicode(s, incodec) LookupError: unknown encoding
I presume this is some Asian encoding that's not *supposed* to appear in an English-language list, but does.
Yep. Here's the start of message #4077:
"""
From mswang@princeton.com.tw Fri Dec 17 06:33:22 1999 To: mems@isi.edu Subject: request info. on GDSII
From: "Tina Wang" <mswang@princeton.com.tw>
Date: Fri Dec 17 06:33:22 1999 Status: O Content-Length: 1330 Lines: 45
This is a multi-part message in MIME format.
------=_NextPart_000_001A_01BF489B.AB8341E0 Content-Type: text/plain; charset="big5" <<< I don't think so Content-Transfer-Encoding: quoted-printable
Dear MEMES Discussion Group, [...] """
Obviously that first message part is *not* encoded with big5, but you try telling Mailman that. Sigh. I'll try editing the offending message and seeing how far arch gets this time.
Oh yeah, should I be filing bug reports?
Greg-- Greg Ward - software developer gward@mems-exchange.org MEMS Exchange http://www.mems-exchange.org
"GW" == Greg Ward <gward@mems-exchange.org> writes:
GW> I presume this is some Asian encoding that's not *supposed* to
GW> appear in an English-language list, but does. Grumble. I'll
GW> see if I can figure out the offending message.Some one else reported this, but I'm still waiting for an example so I can reproduce the bug. If you find it, please send it (or better yet, add it to a SF bug report).
Thanks! -Barry
participants (2)
-
barry＠python.org -
Greg Ward