Character encoding problems with digest option
Hello.
I'm managing several mailing list with mailman 2.1.2, but one of them is especially problematic, since some users use exotics charset such as utf-8. As a matter a consequence, it frequently breaks with this message in the logs: Aug 31 22:03:16 2003 (12355) Traceback (most recent call last): File "/var/lib/mailman/Mailman/Queue/Runner.py", line 105, in _oneloop self._onefile(msg, msgdata) File "/var/lib/mailman/Mailman/Queue/Runner.py", line 155, in _onefile keepqueued = self._dispose(mlist, msg, msgdata) File "/var/lib/mailman/Mailman/Queue/ArchRunner.py", line 73, in _dispose mlist.ArchiveMail(msg) File "/var/lib/mailman/Mailman/Archiver/Archiver.py", line 206, in ArchiveMail h.processUnixMailbox(f) File "/var/lib/mailman/Mailman/Archiver/pipermail.py", line 544, in processUnixMailbox m = mbox.next() File "/usr/lib/python2.2/mailbox.py", line 33, in next return self.factory(_Subfile(self.fp, start, stop)) File "/var/lib/mailman/Mailman/Mailbox.py", line 89, in scrubber return mailbox.scrub(msg) File "/var/lib/mailman/Mailman/Mailbox.py", line 109, in scrub return self._scrubber(self._mlist, msg) File "/var/lib/mailman/Mailman/Handlers/Scrubber.py", line 316, in process t = unicode(t, 'ascii', 'replace').encode('ascii') UnicodeError: ASCII encoding error: ordinal not in range(128)
Searching through the list archive, i found this thread: http://www.mail-archive.com/mailman-users@python.org/msg17270.html
So i applied the patch found at http://sourceforge.net/tracker/?func=detail&aid=670167&group_id=103&atid=300103 i added add_language('en', 'English', 'iso-8859-1') to mm_cfg.py and followed the instruction given:
- restart mailman
- remove digest.mbox
- unshunt
This worked several times already, but this time it seems to be unefficient. Running unshunt trigger the error, even after restarting mailman and check diget.mbxo didn't existed.
Turning ogg digest is an option, however some users are subscribed this way, so i have no way of reaching them to prevent of the policy change.
Any idea ?
Guillaume Rousse The harder it is to obtain matching hardware, the more the customer will insist on an exact match If you have an exact match the customer will say " Matching isn't important, don't you have anything cheaper?" -- Murphy's Laws of Locksmithing n°20
On Aug 31, 2003, at 2:29 PM, Guillaume Rousse wrote:
I'm managing several mailing list with mailman 2.1.2, but one of them is especially problematic, since some users use exotics charset such as utf-8.
utf-8 is exotic?
-- "Queen Isabella and King Ferdinand kicked 200,000 Jews out of Spain, one of the first acts of the Spanish Inquisition, which no one ever expects. " John Carroll's 21st Annual Xmas Quiz answers
Ainsi parlait LuKreme (List User Kreme) :
On Aug 31, 2003, at 2:29 PM, Guillaume Rousse wrote:
I'm managing several mailing list with mailman 2.1.2, but one of them is especially problematic, since some users use exotics charset such as utf-8.
utf-8 is exotic? more than iso-88591-1, and enough to give me some headaches with mailman :-)
Guillaume Rousse Every kind action has a not-so-kind reaction -- Murphy's Love Laws n°8
Hi,
Guillaume Rousse wrote:
File "/var/lib/mailman/Mailman/Handlers/Scrubber.py", line 316, in process t = unicode(t, 'ascii', 'replace').encode('ascii')
Find above line and change to t = unicode(t, 'ascii', 'replace')
Encode will be done more safely in later part.
-- Tokio Kikuchi, tkikuchi@ is.kochi-u.ac.jp http://weather.is.kochi-u.ac.jp/
Ainsi parlait Tokio Kikuchi :
Hi,
Guillaume Rousse wrote:
File "/var/lib/mailman/Mailman/Handlers/Scrubber.py", line 316, in process t = unicode(t, 'ascii', 'replace').encode('ascii')
Find above line and change to t = unicode(t, 'ascii', 'replace')
Encode will be done more safely in later part. Thanks for your reply, it worked.
Does it means that this change is enough to fix the problem, or is your Scrubber.py patch also needed ? In latter case, why does it not also include this part ?
Guillaume Rousse Men's desire for sex sometimes results in intimacy. Women's desire for intimacy often results in sex. -- The Unintended Result
Hi,
Guillaume Rousse wrote:
Thanks for your reply, it worked.
Does it means that this change is enough to fix the problem, or is your Scrubber.py patch also needed ?
You will become happier if you apply the patch.
In latter case, why does it not also include this part ?
It came after I last updated my patch. I will soon include it.
-- Tokio Kikuchi, tkikuchi@ is.kochi-u.ac.jp http://weather.is.kochi-u.ac.jp/
participants (3)
-
Guillaume Rousse -
LuKreme -
Tokio Kikuchi