[Mailman-Users] Fwd: mailpasswds meeses with encoding

[Arkadiusz Miskiewicz]

Since mailman-developers@ moderators are not working (or are very busy) I'm 
sending these here. Btw. it's better to set list to reject non-members posts 
if moderators do not have time - I would not wait and just subscribe there 
but since I'm on -users.... :)

----------  Forwarded Message  ----------

Subject: mailpasswds meeses with encoding
Date: Saturday 03 of July 2004 20:11
From: Arkadiusz Miskiewicz <arekm at pld-linux.org>
To: mailman-developers at python.org

[root at anduril cron]# su - mailman -s /bin/sh -c "python
-S /var/lib/mailman/cron/mailpasswds -l pld-rc-scripts"
Traceback (most recent call last):
  File "/var/lib/mailman/cron/mailpasswds", line 239, in ?
    main()
  File "/var/lib/mailman/cron/mailpasswds", line 214, in main
    text += (header + '\n' + NL.join(table))
UnicodeDecodeError: 'ascii' codec can't decode byte 0xb3 in position 44:
ordinal not in range(128)

the problem is that header is created:
header = '%-40s %-10s\n%-40s %-10s' % (
                _('List'), _('Password // URL'), '----', '--------')

but for example Password in Polish contains non-ascii characters so this
 patch is needed:

 # Coerce everything to Unicode
+header = tounicode(header, enc)
 text = tounicode(text, enc)
 table = [tounicode(_t, enc) for _t in table]

This fixes problem for me.

http://sourceforge.net/tracker/index.php?func=detail&aid=984682&group_id=103&
atid=300103

-------------------------------------------------------

And another one:

Hi,

Polish templates use iso-8859-2 charset. Now mailpasswds converts
that to unicode and

i18n.set_language(poplang)
                msg = Message.UserNotification(
                    addr, siteowner,
                    _('%(host)s mailing list memberships reminder'),
                    text, poplang)

so it calls UserNotification with text in unicode and poplang='pl'.

In UserNotification:

    def __init__(self, recip, sender, subject=None, text=None, lang=None):
        Message.__init__(self)
        charset = None
        if lang is not None:
            charset = Charset(Utils.GetCharSet(lang))

for pl charset will be iso-8859-2

        if text is not None:
            self.set_payload(text, charset)

so it sets payload with unicode text and tells that it's iso-8859-2 charset, 
bad.


        if subject is None:
            subject = '(no subject)'
        self['Subject'] = Header(subject, charset, header_name='Subject',
                                 errors='replace')

exactly the same here

        self['From'] = sender
        if isinstance(recip, ListType):
            self['To'] = COMMASPACE.join(recip)
            self.recips = recip
        else:
            self['To'] = recip
            self.recips = [recip]

That ends with sort-of-quoted-printable unicode text like:
To jest przypomnienie, wysy=142ane raz na miesi=105c, o tym, =17Ce jeste=
=15B cz=142onkiem


It seems that conversion from unicode to specified lang
charset is missing here. Patch attached that works for me.
Comments?

--- Message.py.org      2004-07-03 20:52:47.847479096 +0200
+++ Message.py  2004-07-03 20:48:48.082928848 +0200
@@ -199,10 +199,10 @@
         if lang is not None:
             charset = Charset(Utils.GetCharSet(lang))
         if text is not None:
-            self.set_payload(text, charset)
+            self.set_payload(text.encode(charset.input_charset,'replace'), 
charset)
         if subject is None:
             subject = '(no subject)'
-        self['Subject'] = Header(subject, charset, header_name='Subject',
+        self['Subject'] = 
Header(subject.encode(charset.input_charset,'replace'), charset, 
header_name='Subject',
                                  errors='replace')
         self['From'] = sender
         if isinstance(recip, ListType):

-------------------------------------------------------

Unfortunately this patch breaks things in other cases so the fix is different 
(don't know yet what - didn't have time to play more with this). 

-- 
Arkadiusz Miśkiewicz     CS at FoE, Wroclaw University of Technology
arekm.pld-linux.org, 1024/3DB19BBD, JID: arekm.jabber.org, PLD/Linux