Re: [Mailman-Users] utf-8 error with all lists
Realizing I didn't respond to the list. So scroll down for my response. Then come back here to continue reading.
Now that I have my Python-fu working I'm retesting Mark's suggestion:
webserv:~# python Python 2.7.13 (default, Sep 26 2018, 18:42:22) [GCC 6.3.0 20170516] on linux2 Type "help", "copyright", "credits" or "license" for more information.
import string string.lowercase 'abcdefghijklmnopqrstuvwxyz'
It appears that lowercase does return an apparent properly lowercased string of letters. So, I'm uncertain why the hand-edited code would function differently.
Thanks,
Jason Morrill
IT Director
Child and Family Agency
860-443-2896 x1422
http://www.childandfamilyagency.org/
From: Jason Morrill Sent: Monday, May 13, 2019 10:29 AM To: Mark Sapiro Subject: RE: [Mailman-Users] utf-8 error with all lists
Mark, Thanks for the suggestions and link to the remainder of that prior message thread.
I too get the same results related to Python and lowercase.
Now, I'm not a Python programmer but can certainly stumble through to sort things out. Here's what I'm seeing:
webserv:~# python Python 2.7.13 (default, Sep 26 2018, 18:42:22) [GCC 6.3.0 20170516] on linux2 Type "help", "copyright", "credits" or "license" for more information.
import string lowercase Traceback (most recent call last): File "<stdin>", line 1, in <module> NameError: name 'lowercase' is not defined string
Per your suggestion I modified /var/lib/mailman/Mailman/Archiver/pipermail.py to set lowercase = 'abcdefghijklmnopqrstuvwxyz'
And indeed the issue worked itself out. No more errors and I was able to clear the qfiles/shunt folder too.
Per the Python docs, and your info, the string returned by lowercase should be a-z. And it depends upon the locale of the server. Something I've never purposefully changed.
In Debian it looks at /etc/locale.gen
In that file I see my locale set to be en_US ISO-8859-1
Could that somehow be related to this issue?
Thanks,
Jason Morrill
IT Director
Child and Family Agency
860-443-2896 x1422
http://www.childandfamilyagency.org/
From: Mailman-Users [mailman-users-bounces+morrillj=childandfamilyagency.org@python.org] on behalf of Mark Sapiro [mark@msapiro.net] Sent: Saturday, May 11, 2019 2:13 PM To: mailman-users@python.org Subject: Re: [Mailman-Users] utf-8 error with all lists
On 5/10/19 12:37 PM, Jason Morrill wrote:
Since the beginning of May I've had a problem with all messages going through Mailman. They are being delivered to the lists properly but copies are staying in qfiles/shunt
The errors appearing in the Mailman error.1 log file indicate " May 05 10:16:29 2019 (1167) Uncaught runner exception: 'utf8' codec can't decode byte 0xaa in position 26: invalid start byte ... File "/var/lib/mailman/Mailman/Archiver/pipermail.py", line 63, in fixAuthor while i>0 and (L[i-1][0] in lowercase or UnicodeDecodeError: 'utf8' codec can't decode byte 0xaa in position 26: invalid start byte
May 05 10:16:29 2019 (1167) SHUNTING: 1557065787.914554+25c37ef6e662cc98d41ac9d45855fe04f151e11a
"
This is on Debian 9.6 with Python 2.7.13 and Mailman 2.1.23 , all installed from the Debian package manager.
I've spent several hours hunting around for a potential solution. Then stumbled upon this: https://mail.python.org/pipermail/mailman-users/2019-March/084243.html
If you read the entire thread (which is split in the archive with the second part beginning at https://mail.python.org/pipermail/mailman-users/2019-March/084266.html), you will see that I think the issue is that whatever changed in your system at the beginning of May is causing the Python
from string import lowercase
in /var/lib/mailman/Mailman/Archiver/pipermail.py to import a lowercase object which should be the string 'abcdefghijklmnopqrstuvwxyz', but in fact is longer and has non-ascii accented characters following 'z'.
The OP in that thread couldn't verify that that was the case, but I am convinced that it is.
There doesn't seem to be a Debian patch to mailman 1:2.1.23-1+deb9u4 that would affect this so it is apparently something in Python or the OS.
I am certain that if you start with the base /var/lib/mailman/Mailman/Archiver/pipermail.py and change
from string import lowercase
at line 13 to
lowercase = 'abcdefghijklmnopqrstuvwxyz'
the problem will be fixed.
-- Mark Sapiro mark@msapiro.net The highway is for gamblers, San Francisco Bay Area, California better use your sense - B. Dylan
Mailman-Users mailing list Mailman-Users@python.org https://mail.python.org/mailman/listinfo/mailman-users Mailman FAQ: http://wiki.list.org/x/AgA3 Security Policy: http://wiki.list.org/x/QIA9 Searchable Archives: http://www.mail-archive.com/mailman-users%40python.org/ Unsubscribe: https://mail.python.org/mailman/options/mailman-users/morrillj%40childandfam...
participants (3)
-
Dimitri Maziuk
-
Jason Morrill
-
Mark Sapiro