[Mailman-Users] utf-8 error with all lists

[Jason Morrill]

Since the beginning of May I've had a problem with all messages going through Mailman. They are being delivered to the lists properly but copies are staying in qfiles/shunt

The errors appearing in the Mailman error.1 log file indicate
"
May 05 10:16:29 2019 (1167) Uncaught runner exception: 'utf8' codec can't decode byte 0xaa in position 26: invalid start byte
May 05 10:16:29 2019 (1167) Traceback (most recent call last):
  File "/var/lib/mailman/Mailman/Queue/Runner.py", line 119, in _oneloop
    self._onefile(msg, msgdata)
  File "/var/lib/mailman/Mailman/Queue/Runner.py", line 190, in _onefile
    keepqueued = self._dispose(mlist, msg, msgdata)
  File "/var/lib/mailman/Mailman/Queue/ArchRunner.py", line 77, in _dispose
    mlist.ArchiveMail(msg)
  File "/var/lib/mailman/Mailman/Archiver/Archiver.py", line 214, in ArchiveMail
    h.processUnixMailbox(f)
  File "/var/lib/mailman/Mailman/Archiver/pipermail.py", line 596, in processUnixMailbox
    self.add_article(a)
  File "/var/lib/mailman/Mailman/Archiver/pipermail.py", line 640, in add_article
    author = fixAuthor(article.decoded['author'])
  File "/var/lib/mailman/Mailman/Archiver/pipermail.py", line 63, in fixAuthor
    while i>0 and (L[i-1][0] in lowercase or
UnicodeDecodeError: 'utf8' codec can't decode byte 0xaa in position 26: invalid start byte

May 05 10:16:29 2019 (1167) SHUNTING: 1557065787.914554+25c37ef6e662cc98d41ac9d45855fe04f151e11a

"

This is on Debian 9.6 with Python 2.7.13 and Mailman 2.1.23 , all installed from the Debian package manager.

I've spent several hours hunting around for a potential solution. Then stumbled upon this:
https://mail.python.org/pipermail/mailman-users/2019-March/084243.html

Before making the code edits suggested in that post I was performing a simple test using the 'arch' command.

With the original code the arch command fails with every single list in my system. The error it throws looks like this:

"
Updating HTML for article 66
Pickling archive state into /var/lib/mailman/archives/private/jtest/pipermail.pck
Traceback (most recent call last):
  File "./arch", line 201, in <module>
    main()
  File "./arch", line 189, in main
    archiver.processUnixMailbox(fp, start, end)
  File "/var/lib/mailman/Mailman/Archiver/pipermail.py", line 606, in processUnixMailbox
    self.add_article(a)
  File "/var/lib/mailman/Mailman/Archiver/pipermail.py", line 650, in add_article
    author = fixAuthor(article.decoded['author'])
  File "/var/lib/mailman/Mailman/Archiver/pipermail.py", line 72, in fixAuthor
    while i>0 and (L[i-1][0] in lowercase or
UnicodeDecodeError: 'utf8' codec can't decode byte 0xaa in position 26: invalid start byte
"

With the modified code the arch command works perfectly with every single list and no errors are generated in the log file.

I'm going to leave the hand-edited code in-place for the time being. If anyone would like to work with me to diagnose this mystery problem I'd be happy to assist however I can.


Thanks,

  Jason Morrill

  IT Director

  Child and Family Agency

  860-443-2896 x1422

  http://www.childandfamilyagency.org/