Hi, I found it a bit hard task to follow up the versions of mailman-i18n because pygettext.py output is random order. Following patch and executing the script with `--style sort' option make the output more friendly for translaters, I believe. Cheers, Tokio Kikuchi --- pygettext.py.orig Sat May 19 22:36:57 2001 +++ pygettext.py Sat May 19 22:40:43 2001 @@ -100,6 +100,7 @@ Solaris # File: filename, line: line-number GNU #: filename:line + sort (slightly modified GNU with filename/line order) The style name is case insensitive. GNU style is the default. @@ -288,7 +289,27 @@ # The time stamp in the header doesn't have the same format # as that generated by xgettext... print pot_header % {'time': timestamp, 'version': __version__} - for k, v in self.__messages.items(): + if options.locationstyle == options.SORT: + messages = {} + for k, v in self.__messages.items(): + file_line = [] + i = 0 + for filename, lineno in v: + file_line[i:] = ['#: %s:%05d\n'% (filename,lineno)] + i = i + 1 + file_line.sort() + s = '' + for f in file_line: + s = s + f + messages[s] = k + x = messages.keys() + x.sort() + for s in x: + print s, + print 'msgid', normalize(messages[s]) + print 'msgstr ""\n' + else: + for k, v in self.__messages.items(): if not options.writelocations: pass # location comments are different b/w Solaris and GNU: @@ -336,6 +357,7 @@ # constants GNU = 1 SOLARIS = 2 + SORT = 3 # defaults extractall = 0 # FIXME: currently this option has no effect at all. escape = 0 @@ -351,6 +373,7 @@ options = Options() locations = {'gnu' : options.GNU, 'solaris' : options.SOLARIS, + 'sort' : options.SORT, } # parse options
Tokio Kikuchi wrote:
Hi,
I found it a bit hard task to follow up the versions of mailman-i18n because pygettext.py output is random order. Following patch and executing the script with `--style sort' option make the output more friendly for translaters, I believe.
Stupid me! Yesterday version throws away one or more msgid's when in a same line, keeping only one msgid. Here is a revised one. Tokio --- pygettext.py.orig Sat May 19 22:36:57 2001 +++ pygettext.py Sun May 20 14:23:10 2001 @@ -100,6 +100,7 @@ Solaris # File: filename, line: line-number GNU #: filename:line + sort (slightly modified GNU with filename/line order) The style name is case insensitive. GNU style is the default. @@ -139,6 +140,7 @@ import time import getopt import tokenize +import re __version__ = '1.1' @@ -288,7 +290,28 @@ # The time stamp in the header doesn't have the same format # as that generated by xgettext... print pot_header % {'time': timestamp, 'version': __version__} - for k, v in self.__messages.items(): + if options.locationstyle == options.SORT: + separator = '#%= ' + x = [] + for k, v in self.__messages.items(): + file_line = [] + for filename, lineno in v: + file_line.append('#: %s:%05d\n'% (filename,lineno)) + file_line.sort() + s = '' + for f in file_line: + s = s + f + s = s + separator + k + x.append(s) + x.sort() + sep = re.compile(separator) + for s in x: + r, k = sep.split(s) + print r, + print 'msgid', normalize(k) + print 'msgstr ""\n' + else: + for k, v in self.__messages.items(): if not options.writelocations: pass # location comments are different b/w Solaris and GNU: @@ -336,6 +359,7 @@ # constants GNU = 1 SOLARIS = 2 + SORT = 3 # defaults extractall = 0 # FIXME: currently this option has no effect at all. escape = 0 @@ -351,6 +375,7 @@ options = Options() locations = {'gnu' : options.GNU, 'solaris' : options.SOLARIS, + 'sort' : options.SORT, } # parse options
I found it a bit hard task to follow up the versions of mailman-i18n because pygettext.py output is random order. Following patch and executing the script with `--style sort' option make the output more friendly for translaters, I believe.
This is a very good idea. I think pygettext should always sort the output, without even asking the user; ideally in the same way as xgettext would sort them. It appears that your patch sorts them by file-then-line, so does xgettext. Therefore, I propose that you remove the command line option, and submit your change to sf.net/projects/python. Thanks for contributing, Martin
"MvL" == Martin von Loewis <loewis@informatik.hu-berlin.de> writes:
MvL> This is a very good idea. I think pygettext should always MvL> sort the output, without even asking the user; ideally in the MvL> same way as xgettext would sort them. It appears that your MvL> patch sorts them by file-then-line, so does xgettext. MvL> Therefore, I propose that you remove the command line option, MvL> and submit your change to sf.net/projects/python. Sounds like a good idea. Feel free to assign your patch to me and I'll take it from there. -Barry
"MvL" == Martin von Loewis <loewis@informatik.hu-berlin.de> writes:
>> I found it a bit hard task to follow up the versions of >> mailman-i18n because pygettext.py output is random order. >> Following patch and executing the script with `--style sort' >> option make the output more friendly for translaters, I >> believe. MvL> This is a very good idea. I think pygettext should always MvL> sort the output, without even asking the user; ideally in the MvL> same way as xgettext would sort them. It appears that your MvL> patch sorts them by file-then-line, so does xgettext. MvL> Therefore, I propose that you remove the command line option, MvL> and submit your change to sf.net/projects/python. I'm about to check in a slightly different patch, which retains packed GNU style comments and also fixes a bug that can cause duplicate entries. Details (including the patch) are available at http://sourceforge.net/tracker/index.php?func=detail&aid=425821&group_id=5470&atid=305470 Thanks! -Barry
"Barry A. Warsaw" wrote:
I'm about to check in a slightly different patch, which retains packed GNU style comments and also fixes a bug that can cause duplicate entries. Details (including the patch) are available at
http://sourceforge.net/tracker/index.php?func=detail&aid=425821&group_id=5470&atid=305470
Barry, There was a misunderstanding. Your patch sort the filename/lineno descriptions within each item of translatables. My patch sorts all the entries of them. May be it is because my initial post put the option within `--style' category and none of your fault. pygettext.py with my patch outputs like this
#: bin/add_members:00216 msgid "( blank line )" msgstr ""
#: bin/add_members:00218 msgid "Not a valid email address:" msgstr ""
#: bin/arch:00073 msgid "listname is required" msgstr ""
#: bin/arch:00091 #: bin/change_pw:00100 #: bin/config_list:00101 #: bin/config_list:00192 #: bin/newlang:00084 #: bin/rmlang:00095 msgid "" "No such list \"%(listname)s\"\n" "%(e)s" msgstr ""
With this output, we can find where to make new translations easily when the version ups. I hope you can integrate this idea into your patch. Thank you, Tokio
"TK" == Tokio Kikuchi <tkikuchi@is.kochi-u.ac.jp> writes:
TK> There was a misunderstanding. Your patch sort the TK> filename/lineno descriptions within each item of TK> translatables. My patch sorts all the entries of them. May be TK> it is because my initial post put the option within `--style' TK> category and none of your fault. No, I think I just didn't look close enough at your patch, but I see now what it's doing. Okay, so now I've got another patch to pygettext.py which first sorts the locations in each individual entry (by filename then lineno), and then sorts the catalog entries by the tuples of location. So you end up with entries like the attached, taken from the head of mailman.pot. If that looks good, I'll check it in. -Barry -------------------- snip snip -------------------- #. docstring #: Mailman/Archiver/Archiver.py:18 msgid "" "Mixin class for putting new messages in the right place for archival.\n" "\n" "Public archives are separated from private ones. An external archival\n" "mechanism (eg, pipermail) should be pointed to the right places, to do the\n" "archival.\n" msgstr "" #: Mailman/Archiver/Archiver.py:123 msgid "List traffic archival policies." msgstr "" #: Mailman/Archiver/Archiver.py:125 Mailman/Autoresponder.py:64 #: Mailman/Autoresponder.py:81 Mailman/Bouncer.py:67 Mailman/Digester.py:55 #: Mailman/Digester.py:71 Mailman/Digester.py:93 Mailman/Digester.py:98 #: Mailman/GatewayManager.py:53 Mailman/GatewayManager.py:57 #: Mailman/GatewayManager.py:61 Mailman/MailList.py:511 #: Mailman/MailList.py:521 Mailman/MailList.py:546 Mailman/MailList.py:550 #: Mailman/MailList.py:557 Mailman/MailList.py:567 Mailman/MailList.py:572 #: Mailman/MailList.py:666 Mailman/MailList.py:680 Mailman/MailList.py:691 #: Mailman/MailList.py:694 Mailman/MailList.py:726 Mailman/MailList.py:800 #: Mailman/MailList.py:808 msgid "No" msgstr "" #: Mailman/Archiver/Archiver.py:125 Mailman/Autoresponder.py:64 #: Mailman/Bouncer.py:67 Mailman/Digester.py:55 Mailman/Digester.py:71 #: Mailman/Digester.py:93 Mailman/Digester.py:98 Mailman/GatewayManager.py:53 #: Mailman/GatewayManager.py:57 Mailman/GatewayManager.py:61 #: Mailman/MailList.py:511 Mailman/MailList.py:521 Mailman/MailList.py:546 #: Mailman/MailList.py:550 Mailman/MailList.py:557 Mailman/MailList.py:567 #: Mailman/MailList.py:572 Mailman/MailList.py:666 Mailman/MailList.py:680 #: Mailman/MailList.py:691 Mailman/MailList.py:694 Mailman/MailList.py:726 #: Mailman/MailList.py:800 Mailman/MailList.py:808 msgid "Yes" msgstr "" #: Mailman/Archiver/Archiver.py:126 msgid "Archive messages?" msgstr "" #: Mailman/Archiver/Archiver.py:128 msgid "public" msgstr ""
"Barry A. Warsaw" wrote:
Okay, so now I've got another patch to pygettext.py which first sorts the locations in each individual entry (by filename then lineno), and then sorts the catalog entries by the tuples of location. So you end up with entries like the attached, taken from the head of mailman.pot.
If that looks good, I'll check it in.
It is exactly what I want. Thank you very much, Barry! Tokio
"Barry A. Warsaw" wrote:
"TK" == Tokio Kikuchi <tkikuchi@is.kochi-u.ac.jp> writes:
TK> It is exactly what I want. Thank you very much, Barry!
Cool! I thought it might be so I've already checked it into the Python tree (and will soon copy it over to the Mailman tree).
Ouch! You have just followed my fault. Here is output from new pygettext.py #: ./Mailman/Cgi/admin.py:750 msgid "member address<br>member name" msgstr "" #: ./Mailman/Cgi/admin.py:751 msgid "hide" msgstr "" #: ./Mailman/Cgi/admin.py:752 msgid "not metoo" msgstr "" , whereas admin.py reads usertable.AddRow([Center(h) for h in (_('member address<br>member name'), _('subscr'), _('hide'), _('nomail'), _('ack'), _('not metoo'), _('digest'), _('plain'), _('language'))]) Only one entry per line was kept in the output. sorry, but it must be fixed. Tokio
"TK" == Tokio Kikuchi <tkikuchi@is.kochi-u.ac.jp> writes:
TK> Ouch! You have just followed my fault. Here is output from TK> new pygettext.py Darn, you're right. Okay, I've now got a new version which aggressively sorts the entries. :) E.g. -------------------- snip snip -------------------- #: Mailman/Cgi/admin.py:729 msgid "member address<br>member name" msgstr "" #: Mailman/Cgi/admin.py:730 msgid "hide" msgstr "" #: Mailman/Cgi/admin.py:730 msgid "nomail" msgstr "" #: Mailman/Cgi/admin.py:730 msgid "subscr" msgstr "" #: Mailman/Cgi/admin.py:731 msgid "ack" msgstr "" #: Mailman/Cgi/admin.py:731 msgid "not metoo" msgstr "" -------------------- snip snip -------------------- To be checked into both Python and Mailman momentarily... -Barry
If that looks good, I'll check it in.
That is indeed what sorting should do, IMO. While sorting and normalization certainly helps when a new version of the catalog comes along, there is actually tool for merging an old localized catalog with the new POT file, in the form of GNU msgmerge. I don't know whether you are all aware of it, so I thought I'd mention it. Is anybody interested in having a pure-python version of msgmerge, or is using GNU msgmerge good enough? Regards, Martin
"MvL" == Martin von Loewis <loewis@informatik.hu-berlin.de> writes:
MvL> While sorting and normalization certainly helps when a new MvL> version of the catalog comes along, there is actually tool MvL> for merging an old localized catalog with the new POT file, MvL> in the form of GNU msgmerge. I don't know whether you are MvL> all aware of it, so I thought I'd mention it. Definitely! In fact the "make catalogs" target in Mailman's messages/Makefile invokes msgmerge to merge any new definitions found in mailman.pot with the individual language .po files. (I hope I understand the usage properly. It looks good to me but others may want to double check.) MvL> Is anybody interested in having a pure-python version of MvL> msgmerge, or is using GNU msgmerge good enough? GNU's msgmerge seems fine to me. Mailman's Makefile is also using the default msgfmt, although I know you contributed one for Python's Tools/i18n directory. If you've got a msgmerge laying around, I think it would make a good addition to Tools/i18n, but it isn't as urgent as pygettext (which was required because the standard GNU tool, xgettext, just didn't grok Python code well enough). -Barry
Martin, Barry, Thank you both for all the help for easier handling of language catalogs. I haven't tried msgmerge before but I now know its power. Martin von Loewis wrote:
If that looks good, I'll check it in.
That is indeed what sorting should do, IMO.
I really needed this feature because human language is very much dependent of context that I must frequently refer the original code.
While sorting and normalization certainly helps when a new version of the catalog comes along, there is actually tool for merging an old localized catalog with the new POT file, in the form of GNU msgmerge. I don't know whether you are all aware of it, so I thought I'd mention it.
Yes, and it tells me the changed string with "#, fussy" mark. Great! And, I put up my Japanese translations of POT and templates at http://mm.tkikuchi.net/mailman-2.1alpha.ja/ These will be updated frequently but I will be happy if you can include them in the next alpha release. Many thanks, Tokio
participants (3)
-
barry＠digicool.com -
Martin von Loewis -
Tokio Kikuchi