[Mailman-Users] Dealing with multiple charsets (list messages and web archive)

[Mark Sapiro]

Stefan Förster wrote:

>* Mark Sapiro <mark at msapiro.net> wrote:
>
>> Also, have you tried setting Non-digest options -> scrub_nondigest to
>> Yes? This may not be satisfactory for you, but if it is, it may help
>> with some of your other issues.
>
>After converting the message catalogues (messages/de/mailman.po) and
>the template files (templates/de/*.{html,txt}) as described in
>README-I18N.en and migrating the lists in question, rebuilding the web
>archives etc., I've switched that option on.
>
>The result is somewhat unexpected: The message translation for
>
>'-------------- next part --------------'
>
>is
>
>'-------------- n"achster Teil --------------'
>
>where the '"a" represents an "Umlaut", "ä". This "ä" gets encoded
>wrongly in the mail sent out, typcial "two chars" error ("Â~" or
>soemthing like that). Interstingly, the next line talking about binary
>data which got cut out, which does _also_ include an "ä" gets encoded
>correctly. Can you give me any hint on this? Replacement is done in
>Scrubber.py, line 392 here:
>
>        # Now join the text and set the payload
>        sep = _('-------------- next part --------------\n')
>        replace_payload_by_text(msg, sep.join(text), charset)


The above problem is a bug fixed in 2.1.10. The fixed code is:

        # Now join the text and set the payload
        sep = _('-------------- next part --------------\n')
        # The i18n separator is in the list's charset. Coerce it to the
        # message charset.
        try:
            s = unicode(sep, lcset, 'replace')
            sep = s.encode(charset, 'replace')
        except (UnicodeError, LookupError, ValueError,
                AssertionError):
            pass
        replace_payload_by_text(msg, sep.join(text), charset)



>Do you think there might be other occurances of wrongly encoded
>strings read in from the message catalogue or the templates?


Not that I'm aware of.

-- 
Mark Sapiro <mark at msapiro.net>        The highway is for gamblers,
San Francisco Bay Area, California    better use your sense - B. Dylan