Does mailman 2.1.18 support hebrew language ?

Mark,
We have a customer who reports one problem - digest is changing hebrew language to question marks. We just want to know whether mailman 2.1.18 support hebrew language. If support, how to set up?
Thanks,
Carl Xie Ohio State University

On 01/09/2015 06:30 AM, Xie, Wei wrote:
Mark,
We have a customer who reports one problem - digest is changing hebrew language to question marks. We just want to know whether mailman 2.1.18 support hebrew language. If support, how to set up?
Mailman has supported Hebrew since 2.1.10. That is not your issue. Your issue is that users are posting Hebrew to a list whose preferred_language is English and Mailman's character set for English is ASCII. The plain text digest is encoded in the character set of the list's preferred language, thus the non-ascii characters are replaced.
You have a number of choices:
- If users subscribe to the MIME format digest, each message will be in
a separate message/rfc822 MIME part and will retain its original encoding.
- If you set the list's preferred_language to Hebrew on the list admin
Language options page, the plain text digest will be utf-8 encoded and the Hebrew characters won't be replaced, but then the web UI and other things like the digest boiler plate and TOC will be in Hebrew.
- You can set Mailman's character set for English to utf-8 by putting
add_language('en', 'English (USA)', 'utf-8')
in mm_cfg.py (and restarting Mailman). The downside of this is the bodies of Mailman generated messages including plain digests will be base64 encoded and will not be readable by non-MIME aware MUAs. This is not much of an issue these days as such MUAs are rare.

Mark,
First, thank for your explanation in detail.
Second, Let me tell you what current settings related to digest.
In the member option page http://WEBSERVER/mailman/options/LISTNAME/<this user>, there is one setting "Get MIME or Plain Text Digests?", current setting is "Plain Text"; In same page, the setting " What language do you prefer?" is "English(USA)" but there is a drop-down list including language "Hebrew";
In the list admin Language options page, the setting "default language for this list" is set to "English(USA)", the setting "Languages supported by this list" is set to "English (USA)" and "Hebrew"; In same page, the setting " When receiving digests, which format is default?" is "Plain", but I see the note "(Edit mime is default digest)";
In file Defaults.py, I see the following setting about English (USA). But in mm_cfg.py, there is no customized setting about English (USA).
add_language('en', _('English (USA)'), 'us-ascii', 'ltr')
Third, the below is my questions about three choices.
If users subscribe to the MIME format digest, each message will be in a separate message/rfc822 MIME part and will retain its original encoding.
Questions - do you mean this customer can change setting "Get MIME or Plain Text Digests?" to "MIME" in above member options page I mention? No need to change other customer settings? Can the setting ensure this customer is able to read English and Hebrew in same message
If you set the list's preferred_language to Hebrew on the list admin Language options page, the plain text digest will be utf-8 encoded and the Hebrew characters won't be replaced, but then the web UI and other things like the digest boiler plate and TOC will be in Hebrew.
Questions - if we make above change, other digest users in this list will see Hebrew, right? The list owners must not agree to this change. Also, do you think we should change the setting the " When receiving digests, which format is default?" from "Plain" to "MIME".
You can set Mailman's character set for English to utf-8 by putting
add_language('en', 'English (USA)', 'utf-8')
in mm_cfg.py (and restarting Mailman). The downside of this is the bodies of Mailman generated messages including plain digests will be base64 encoded and will not be readable by non-MIME aware MUAs. This is not much of an issue these days as such MUAs are rare.
If the settings in above member option page can fix this customer's problem, we will be cautious to make the change to impact our mailing lists.
Thanks,
Carl Xie
-----Original Message----- From: Mark Sapiro [mailto:mark@msapiro.net] Sent: Friday, January 09, 2015 11:22 AM To: Xie, Wei; mailman-users@python.org Subject: Re: Does mailman 2.1.18 support hebrew language ?
On 01/09/2015 06:30 AM, Xie, Wei wrote:
Mark,
We have a customer who reports one problem - digest is changing hebrew language to question marks. We just want to know whether mailman 2.1.18 support hebrew language. If support, how to set up?
Mailman has supported Hebrew since 2.1.10. That is not your issue. Your issue is that users are posting Hebrew to a list whose preferred_language is English and Mailman's character set for English is ASCII. The plain text digest is encoded in the character set of the list's preferred language, thus the non-ascii characters are replaced.
You have a number of choices:
If users subscribe to the MIME format digest, each message will be in a separate message/rfc822 MIME part and will retain its original encoding.
If you set the list's preferred_language to Hebrew on the list admin Language options page, the plain text digest will be utf-8 encoded and the Hebrew characters won't be replaced, but then the web UI and other things like the digest boiler plate and TOC will be in Hebrew.
You can set Mailman's character set for English to utf-8 by putting
add_language('en', 'English (USA)', 'utf-8')
in mm_cfg.py (and restarting Mailman). The downside of this is the bodies of Mailman generated messages including plain digests will be base64 encoded and will not be readable by non-MIME aware MUAs. This is not much of an issue these days as such MUAs are rare.
-- Mark Sapiro <mark@msapiro.netmailto:mark@msapiro.net> The highway is for gamblers, San Francisco Bay Area, California better use your sense - B. Dylan

On 01/09/2015 09:41 AM, Xie, Wei wrote:
- In the member option page
_http://WEBSERVER/mailman/options/LISTNAME/<this_ user>, there is one setting "Get MIME or Plain Text Digests?", current setting is "Plain Text"; In same page, the setting " What language do you prefer?" is "English(USA)" but there is a drop-down list including language "Hebrew";
OK. Note that this user could set her preferred language to Hebrew, but that wont help solve this issue.
- In the list admin Language options page, the setting "default
language for this list" is set to "English(USA)", the setting "Languages supported by this list" is set to "English (USA)" and "Hebrew";
OK
In same page, the setting " When receiving digests, which format is
default?" is "Plain", but I see the note "(Edit mime is default digest)";
I think that would be on the Digest options page, not the Language options page. The name of the list attribute (setting) is 'mime_is_default_digest' It has two possible values, False indicating Plain is the default and True indicating MIME is the default.
Every setting in the web UI has a link associated with it which is either "Details for setting_name" (a link to additional info) or if there is no additional info, "Edit setting_name" which just links to a page containing only that setting.
I.E. 'mime_is_default_digest' is the name of the setting, not the actual default.
- In file Defaults.py, I see the following setting about English (USA).
But in mm_cfg.py, there is no customized setting about English (USA).
add_language('en', _('English (USA)'), 'us-ascii', 'ltr')
Yes, and if you wanted to change that default you could put, e.g.
add_language('en', 'English (USA)', 'utf-8', 'ltr')
in mm_cfg.py (never change Defa>
ults.py). For technical reasons, if you override this setting in
mm_cfg.py, you should not include the i18n _() function around the language name. Also, the language direction 'ltr' -> left to right is the default and can be omitted which Is why it would be sufficient to put
add_language('en', 'English (USA)', 'utf-8')
in mm_cfg.py to change Mailman's character set for 'en'. Note also that this only works because the English templates and messages are in ASCII which is a proper subset of utf-8.
Third, the below is my questions about three choices.
- If users subscribe to the MIME format digest, each message will be in a separate message/rfc822 MIME part and will retain its original encoding.
Questions – do you mean this customer can change setting "Get MIME or Plain Text Digests?" to “MIME” in above member options page I mention? No need to change other customer settings?
Yes.
Can the setting ensure this customer is able to
read English and Hebrew in same message
I have no idea what the user can read, but with this setting and no change to the list's preferred_language, the digest masthead and TOC will still be in English, and each message in the digest will be in its own message/rfc822 part in the original language and character set in which it was posted, just like the messages delivered to non-digest members.
Note however that some MUAs (mail clients) are not good at understanding MIME format digests so the user experience will vary depending on their choice of MUA. The only way for a user to know if she will like the MIME format is to try it.
- If you set the list's preferred_language to Hebrew on the list admin Language options page, the plain text digest will be utf-8 encoded and the Hebrew characters won't be replaced, but then the web UI and other things like the digest boiler plate and TOC will be in Hebrew.
Questions – if we make above change, other digest users in this list will see Hebrew, right? The list owners must not agree to this change. Also, do you think we should change the setting the " When receiving digests, which format is default?" from "Plain" to “MIME”.
Yes, I think for this list in particular, 'mime_is_default_digest' should be set to MIME.
If you change the list's preferred language to Hebrew, the list's admin and admindb interfaces will be in Hebrew, the digest manifest and TOC will be in Hebrew, Mailman's notices to the admins/moderators will be in Hebrew, but Mailman notices (not digests) to users will be in the user's preferred language as set on the options page.
The individual messages in the digest will be in whatever language they were originally posted, but the character set of the plain format digest will be utf-8 so that Hebrew letters won't be changed to '?'
- You can set Mailman's character set for English to utf-8 by putting
add_language('en', 'English (USA)', 'utf-8')
in mm_cfg.py (and restarting Mailman). The downside of this is the bodies of Mailman generated messages including plain digests will be base64 encoded and will not be readable by non-MIME aware MUAs. This is not much of an issue these days as such MUAs are rare.
If the settings in above member option page can fix this customer’s problem, we will be cautious to make the change to impact our mailing lists.
Set the users to receive MIME digests and set MIME as the default format. There is now a script at http://www.msapiro.net/scripts/set_mime.py that will do that.
participants (2)
-
Mark Sapiro
-
Xie, Wei