On Monday 07 October 2002 14:42, Martin v. Löwis wrote:

Arash Zeini writes:

...

I am new to this list. As far as I could see from the archives, there is no development toward support for RTL and BiDi for languages like Farsi and Arabic. Is this correct?

What kind of support would you expect? From a mailman point of view, this issue does not exist, AFAICT.

I guess the problem may not exist if I post to a mailing list from within an email client that supports Farsi. OK, there might not be an issue. But the problem may occur as soon as we talk about the archives, where HTML is generated. My guess is everything will be left-to-right and unreadable. This is why I posted. While managing a mailing list I tried to enter some description text in Farsi. Farsi, Arabic and Hebrew are written right-to-left and all descriptions that I entered occurred just LTR. Also the generated HTML in mailman does not use UTF-8 encoding why again we have a problem. I.e. no browser would display any of these three languages correctly. One has to set it manually to be able to read the text. Finding a solution shouldn't be too complicated, as all of this is supported through HTML.

Now, translating mailman to Arabic, Hebrew, Farsi languages is another story - contributions are certainly welcome.

You are right, this is another story. But even here we need the proper structure in the HTML pages to be able to translate anything. We need everything mirrored otherwise it is not useable. This is done through the dir="RTL" attribute in HTML. And we need UTF-8 as encoding in the header part of the HTML page. In KDE this problem is solved this way: in the main .pot file there is one string which is not a translatable string but the entry which denotes the text direction. I.e. the translator fills in the value RTL or LTR and according to this value the interface and text direction is set. The same could be done in mailman I guess. Now, if we have the time to work on the translation of Mailman so soon, that is_another_story :) Greetings, Arash -- The FarsiKDE Project http://www.farsikde.org

On Tuesday 08 October 2002 13:23, Martin v. Löwis wrote:

Arash Zeini writes:

...

But the problem may occur as soon as we talk about the archives, where HTML is generated. My guess is everything will be left-to-right and unreadable.

Did you try? My guess is that it comes out correctly, since the Web browser will do RTL.

No, I didn't try. Usually the web browser doesn't do the RTL automatically, if this is what you mean. Wouldn't it need the dir attribute? I guess the best way to try is to have a test mailman 2.1. Is it worth it to try on older versions? And is there such a testing installation anywhere, or should i set one for myself?

...

This is why I posted. While managing a mailing list I tried to enter some description text in Farsi. Farsi, Arabic and Hebrew are written right-to-left and all descriptions that I entered occurred just LTR.

When entering them, or when Mailman was displaying them back?

While Mailman was displaying them back.

...

Also the generated HTML in mailman does not use UTF-8 encoding why again we have a problem.

What Mailman version? I don't think UTF-8 is strictly necessary - wouldn't ISO-8859-6 work as well?

UTF-8 is the recommended one for Farsi. ISO-8859-6 works best for Arabic.

...

I.e. no browser would display any of these three languages correctly. One has to set it manually to be able to read the text.

Did you configure Mailman to change the encoding of the pages?

No. I was not aware of this feature. Is UTF-8 a supported encoding. And please note that I am not a Mailman expert. I encountered a problem and contacted the list to make suggestions.

...

You are right, this is another story. But even here we need the proper structure in the HTML pages to be able to translate anything. We need everything mirrored otherwise it is not useable. This is done through the dir="RTL" attribute in HTML.

I think this can go into the page templates.

I had a look at the distribution package and understand the structure better now. True the templates would be the easiest place.

...

in the main .pot file there is one string which is not a translatable string but the entry which denotes the text direction. I.e. the translator fills in the value RTL or LTR and according to this value the interface and text direction is set. The same could be done in mailman I guess.

I think it can be solved in a much simpler way.

What would be the recommended way? Arash -- The FarsiKDE Project http://www.farsikde.org

On Tuesday 08 October 2002 15:28, Martin v. Löwis wrote:

Arash Zeini writes:

...

No, I didn't try. Usually the web browser doesn't do the RTL automatically, if this is what you mean. Wouldn't it need the dir attribute?

I've tried both MSIE6 and Mozilla/Gecko, and both render Arabic text RTL without a dir attribute. They just know, from looking at their Unicode database, what directionality each character has.

I think we are talking about two different things. Sure the browsers display RTL correctly on the word level. The characters are ordered correctly, but the sentence as a total or even the word itself is not placed on the right side of the browser. Neither forms nor fields are mirrored by the browser if the dir attribute is missing. Also the scrollbar is not placed on the left side without the necessary HTML attibutes. (At least to judge from Konqueror and Mozilla, donot know about IE) I am attaching a sample HTML file to demonstrate this. The socon para doesn't have the dir attribute and hence is not positioned correctly and the flow of the sentence is not correct as well.

IMHO, the HTML dir attribute was invented by people who did not think it possible to implement such a BiDi algorithm in the Web browser.

No, the dir attribute is essential, IMHO.

...

I guess the best way to try is to have a test mailman 2.1. Is it worth it to try on older versions?

In this area, I recommend to use 2.1bsomething.

...

And is there such a testing installation anywhere, or should i set one for myself?

I don't know of any test installation for such purposes.

...

UTF-8 is the recommended one for Farsi. ISO-8859-6 works best for Arabic.

I see. You should add those to Mailman/Defaults.py.

You mean in CVS or locally?

I'd recommend to add Farsi as a language to mailman, and take no additional steps for RTL. I still think it would work out just fine.

As I said I doubt this would be enough by its own. I need to make some further investigation adn will try to install a local Mailman to test things. I have the latest 2.1bsomething :) Greetings, Arash -- The FarsiKDE Project http://www.farsikde.org

On Tuesday 08 October 2002 16:23, Martin v. Löwis wrote: [...]

Good! It may be that some people here are more familiar with Farsi, Arabic, or Hebrew than I am - but don't expect too much help on RTL issues. When it comes to Python problems, don't hesitate to ask for advise and help.

Good, thanks for the help and discussion. I will get back to the list once I have a more practical approach and know what exactly would/could be missing. Greetings, Arash -- The FarsiKDE Project http://www.farsikde.org

On Tuesday 08 October 2002 16:30, Barry A. Warsaw wrote:

...

...

...

...

...

"AZ" == Arash Zeini writes:

AZ> I am attaching a sample HTML file to demonstrate this. The AZ> socon para doesn't have the dir attribute and hence is not AZ> positioned correctly and the flow of the sentence is not AZ> correct as well.

I didn't see any attachments. -Barry

Ahhhhhhhhhh. Here goes the attachment. Greetings Arash -- The FarsiKDE Project http://www.farsikde.org

On Tuesday 08 October 2002 20:55, Barry A. Warsaw wrote:

Near as I can tell, this looks great to me in Moz 1.1.

-Barry

OK, my mistake. I should have choosen a shorter sentence to demonstrate that it sticks to the left. Attached is a new example. Arash -- The FarsiKDE Project http://www.farsikde.org

...

...

...

...

"MvL" == Martin v Löwis writes:

>> Near as I can tell, this looks great to me in Moz 1.1. MvL> I think Arash is pointing out that the first paragraph MvL> right-adjusts in the Window, whereas the second paragraph MvL> left-adjusts. MvL> If you could read Farsi, you would probably also notice that MvL> the order of words is incorrect in the second paragraph, MvL> since the first word is on the left end of the line, so if MvL> you read RTL, you start in the middle of a sentence MvL> (depending on where your browser breaks the lines). I see that now, thanks. Boy, Martin, I didn't know you read Farsi too! :) MvL> In the first paragraph, all is fine: the first word is on the MvL> right end of the first line, and stays there no matter how MvL> you resize the window. This is caused by the dir="rtl" of the MvL> P element. MvL> So, in short, you do need the dir attribute - the browser MvL> does not automatically set the directionality of the MvL> paragraph. MvL> For Mailman, this means you need to augment all templates MvL> appropriately - I guess this is usually done on the <html> MvL> tag. If you ever generate the HTML tag without a MvL> customization hook (e.g. as in htmlformat.Document.Format), MvL> then this might cause a problem: you will need to inject MvL> "DIR='RTL'" there somehow. Alternatively, you'll have to move MvL> the dir attribute further down, to, say, TITLE and BODY. The templates should be easy, since you'd just add those to the fa version. Here's a sketch of what Arash might want to hack together for htmlformat.Document.Format(): - Add a dictionary to Defaults.py.in called LC_DIRECTIONS, where the key is a language code and the value is a flag. I don't want to change the interface for LC_DESCRIPTIONS. - Add a function to Utils.py that returns the direction for a given language code by looking up the language in LC_DIRECTIONS. The default should be LTR so if there's no key in the dict, it's LTR. - Add the appropriate dir attribute to the appropriate tag in Format() based on the lookup. You probably should not add a dir attribute for LTR. That may be all you need, although check the archiver for other places where <html> might be hardcoded. -Barry

...

...

...

...

"MvL" == Martin v Löwis writes:

MvL> I'd recommend to add Farsi as a language to mailman, and take MvL> no additional steps for RTL. I still think it would work out MvL> just fine. I concur with Martin. I would recommend using Mailman 2.1 from cvs. I'm going to be working toward beta4 this week, so expect checkins and fixes. If you're not comfortable with cvs, then definitely use MM2.1b3. You should be able to follow the instructions in README-I18N.en and the examples in Defaults.py.in, the templates directory, and the messages directory to figure out what you'd need to add. It should be fairly straightforward to get the infrastructure in place, and this list can certainly help with details. As for whether Pipermail can handle it, that's a good question. There's a patch on SF for improving the i18n of Pipermail which I intend to look at today: http://sf.net/tracker/index.php?func=detail&aid=594771&group_id=103&atid=300103 Just be aware that time is running out for MM2.1. I'm going to get this sucker out by the end of the year (if not before) if I have to setup a caffeine IV drip and and hack in the shower. :) If the BiDi changes are extensive -- and I hope/expect they won't be -- we'll have to postpone them to the next release. -Barry

...

...

...

...

"AZ" == Arash Zeini writes:

AZ> Again no problem. I would also strongly recommend that you AZ> take in KBabel into the README file, as I think that working AZ> POT concept without KBabel is a mental turtor :) I may be able AZ> to give you some instructions to include in the README file on AZ> how to use KBabel. It is defenitly the supreme way of POTing AZ> :) Of course, I'm an old emacshead so po-mode works well for me, but I just fired up KBabel and it seems nice too. Then again, I'm a monolinguist (various high school and college language courses notwithstanding) so I don't actually /do/ much translating. I've added a short reference to it in the README-I18N.en file, but anything more you can provide will be appreciated. AZ> OK, here I would say don't wait for us. We are very busy with AZ> the translation of KDE and other farsification issues, but we AZ> will try our Best to get this thing going as soon as possible. Excellent. I fully expect there to be micro releases of the 2.1 branch and I wouldn't have too much qualms adding non-intrusive support for Farsi in a micro release. AZ> I will keep you and the list updated. I need to make some test AZ> first. Great, thanks. -Barry