Re: [Mailman-Users] Uncaught runner exception
On 3/8/19 8:17 AM, Mark Sapiro wrote:
On 3/8/19 12:05 AM, Lothar Schilling wrote:
Am 07.03.2019 um 23:54 schrieb Mark Sapiro:
I would still like to find the underlying issue. I have been unable to duplicate it locally. I would appreciate it if before you run bin/unshunt, you could copy a few of the .pck files aside and send them to me off-list. Will send some .pck files to your private address. Some may hold content which is not meant to be publicly available, they are for your eyes only :-)
Thank you. I will treat them confidentially.
I have set up a test wherein I symlinked the qfiles path from the metadata in the shunted .pck files to the qfiles directory in the test installation, created the 5 lists that the shunted messages were destined to with preferred language = German and unshunted the 5 messages.
The test Mailman is the head of the https://code.launchpad.net/~mailman-coders/mailman/2.1 branch, but this differs from the 2.1.29 release in minor ways, none of which have to do with archiving.
Each of the unshunted messages was properly archived with no exceptions being thrown and qfiles/shunt/ was empty after unshunting.
I then removed the 5 lists and started over, this time with
ARCHIVER_OBSCURES_EMAILADDRS = No
which is the only setting I see offhand that might affect this, but the result was the same, no error.
Also, I note that some of the shunted messages had no non-ascii in From:
I don't know what's going on in your installation, but I can't duplicate the issue. It may have something to do with list configuration or your German message catalog or templates if you have any changes in that area (i.e. any templates/site/de/* templates/HOST_NAME/de/* or templates/lists/LIST_NAME/de/* files not properly encoded as iso-8859-1). It could also be something in the archives/private/LIST_NAME/database/* files, although this would probably only affect the period in which you upgraded.
In order to go any further with this, I'd need to see complete error and traceback information from Mailman's error log for the recently shunted messages.
An example of a complete error message:
Mar 11 17:02:22 2019 (15400) SHUNTING: 1552320069.504329+15fb4f71a5404fd1af3dafbe2780486d5b474cc7 Mar 11 17:22:37 2019 (15400) Uncaught runner exception: 'ascii' codec can't decode byte 0xb5 in position 26: ordinal not in range(128) Mar 11 17:22:37 2019 (15400) Traceback (most recent call last): File "/usr/lib/mailman/Mailman/Queue/Runner.py", line 119, in _oneloop self._onefile(msg, msgdata) File "/usr/lib/mailman/Mailman/Queue/Runner.py", line 190, in _onefile keepqueued = self._dispose(mlist, msg, msgdata) File "/usr/lib/mailman/Mailman/Queue/ArchRunner.py", line 77, in _dispose mlist.ArchiveMail(msg) File "/usr/lib/mailman/Mailman/Archiver/Archiver.py", line 216, in ArchiveMail h.processUnixMailbox(f) File "/usr/lib/mailman/Mailman/Archiver/pipermail.py", line 602, in processUnixMailbox self.add_article(a) File "/usr/lib/mailman/Mailman/Archiver/pipermail.py", line 646, in add_article author = fixAuthor(article.decoded['author']) File "/usr/lib/mailman/Mailman/Archiver/pipermail.py", line 63, in fixAuthor while i>0 and (L[i-1][0] in lowercase or UnicodeDecodeError: 'ascii' codec can't decode byte 0xb5 in position 26: ordinal not in range(128)
The problem is site-wide, not just for one or two lists. Anyway, there are no altered templates or messages for a specific list. The only files deviating from the default configuration are: Defaults.py mm_cfg.py
Of course Defaults.py has DEFAULT_SERVER_LANGUAGE = 'en'.
So in mm_cfg.py I added DEFAULT_SERVER_LANGUAGE = 'de' add_language('en', 'English (USA)', 'iso-8859-1', 'ltr')
But I cannot see how this would cause the problem.
On 3/12/19 2:37 AM, Lothar Schilling wrote:
An example of a complete error message:
Mar 11 17:02:22 2019 (15400) SHUNTING: 1552320069.504329+15fb4f71a5404fd1af3dafbe2780486d5b474cc7 Mar 11 17:22:37 2019 (15400) Uncaught runner exception: 'ascii' codec can't decode byte 0xb5 in position 26: ordinal not in range(128) Mar 11 17:22:37 2019 (15400) Traceback (most recent call last): File "/usr/lib/mailman/Mailman/Queue/Runner.py", line 119, in _oneloop self._onefile(msg, msgdata) File "/usr/lib/mailman/Mailman/Queue/Runner.py", line 190, in _onefile keepqueued = self._dispose(mlist, msg, msgdata) File "/usr/lib/mailman/Mailman/Queue/ArchRunner.py", line 77, in _dispose mlist.ArchiveMail(msg) File "/usr/lib/mailman/Mailman/Archiver/Archiver.py", line 216, in ArchiveMail h.processUnixMailbox(f) File "/usr/lib/mailman/Mailman/Archiver/pipermail.py", line 602, in processUnixMailbox self.add_article(a) File "/usr/lib/mailman/Mailman/Archiver/pipermail.py", line 646, in add_article author = fixAuthor(article.decoded['author']) File "/usr/lib/mailman/Mailman/Archiver/pipermail.py", line 63, in fixAuthor while i>0 and (L[i-1][0] in lowercase or UnicodeDecodeError: 'ascii' codec can't decode byte 0xb5 in position 26: ordinal not in range(128)
Again, this is the code trying to "Canonicalize a name into Last, First format". The complete statement is
while i>0 and (L[i-1][0] in lowercase or
L[i-1].lower() in smallNameParts):
i = i - 1
smallNameParts is all lower case ascii, i.e.
= ['van', 'von', 'der', 'de']
and L is the list returned by
L = author.split()
and none of the shunted .pck files you provided me have any name pieces longer than 14 bytes and only 2 of the 5 even have any non-ascii, but the decode error refers to byte 0xb5 in position 26 (B5 in iso-8859-1 is ยต).
I know you said in your original post
I am using Python 2.6.6 on a Centos 6 system. The output of /usr/lib64/python >>> import string >>> string.lowercase is 'abcdefghijklmnopqrstuvwxyz' (as it is supposed to be).
but see below.
The problem is site-wide, not just for one or two lists. Anyway, there are no altered templates or messages for a specific list. The only files deviating from the default configuration are: Defaults.py mm_cfg.py
Of course Defaults.py has DEFAULT_SERVER_LANGUAGE = 'en'.
So in mm_cfg.py I added DEFAULT_SERVER_LANGUAGE = 'de' add_language('en', 'English (USA)', 'iso-8859-1', 'ltr')
But I cannot see how this would cause the problem.
Neither can I, but I tried again with those settings just to be sure, and got no issues.
All of this leads me to conclude that the issue must be non-ascii following the 'z' in string.lowercase.
What is the path to the Python interpreter that Mailman is using?
What do you get if in that Python you do
import string lowercase string
For example, I get
import string lowercase 'abcdefghijklmnopqrstuvwxyz' string
-- Mark Sapiro mark@msapiro.net The highway is for gamblers, San Francisco Bay Area, California better use your sense - B. Dylan
participants (2)
-
Lothar Schilling
-
Mark Sapiro