[ python-Bugs-1158490 ] locale fails if LANGUAGE has multiple locales

Mon Oct 17 14:06:02 CEST 2005

Bugs item #1158490, was opened at 2005-03-07 20:11
Message generated for change (Comment added) made by ber
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1158490&group_id=5470

Please note that this message will contain a full copy of the comment thread,
including the initial issue submission, for this request,
not just the latest update.
Category: Python Library
Group: Python 2.4
Status: Open
Resolution: None
Priority: 3
Submitted By: mixedpuppy (mixedpuppy)
Assigned to: M.-A. Lemburg (lemburg)
Summary: locale fails if LANGUAGE has multiple locales

Initial Comment:
The locale module does not correctly handle the
LANGUAGE environment variable if it contains multiple
settings.  Example:

LANGUAGE="en_DK:en_GB:en_US:en"

Note, en_DK does not exist in locale_alias

In normalize, the colons are replaced with dots, which
is incorrect.  getdefaultlocal should seperate these
first, then try each one until it finds one that works,
or fails on all.  

GLIBC documentation:
http://www.delorie.com/gnu/docs/glibc/libc_138.html

"While for the LC_xxx variables the value should
consist of exactly one specification of a locale the
LANGUAGE variable's value can consist of a colon
separated list of locale names."

Testing this is simple, just set your LANGUAGE
environment var to the above example, and use
locale.getdefaultlocal()

> export LANGUAGE="en_DK:en_GB:en_US:en"
> python
ActivePython 2.4 Build 244 (ActiveState Corp.) based on
Python 2.4 (#1, Feb  9 2005, 19:33:15)
[GCC 3.3.1 (SuSE Linux)] on linux2
Type "help", "copyright", "credits" or "license" for
more information.
>>> import locale
>>> locale.getdefaultlocale()
Traceback (most recent call last):
  File "<stdin>", line 1, in ?
  File "/opt/ActivePython-2.4/lib/python2.4/locale.py",
line 344, in getdefaultlocale
    return _parse_localename(localename)
  File "/opt/ActivePython-2.4/lib/python2.4/locale.py",
line 278, in _parse_localename
    raise ValueError, 'unknown locale: %s' % localename
ValueError: unknown locale: en_DK:en_GB:en_US:en
>>>

----------------------------------------------------------------------

Comment By: Bernhard Reiter (ber)
Date: 2005-10-17 14:06

Message:
Logged In: YES 
user_id=113859

Hi, 

using other information first seems to be a step forward to me. 
I just could not see this from the given example. 

But if LANGUAGE will be evaluated, will the colon be parsed correctly 
and the results tested? 
This seems to be the remainder of this bug. 

Bernhard R. 

----------------------------------------------------------------------

Comment By: M.-A. Lemburg (lemburg)
Date: 2005-10-17 11:30

Message:
Logged In: YES 
user_id=38388

Hi Bernhard,

sorry my last comment wasn't clear: you get this output if
you set the LANGUAGE variable to the example you gave
(LANGUAGE=pt_BR:pt_PT:pt).

The parsing order was changed, so that LANGUAGE is no longer
searched for first, but instead as last resort if the other
locale variables are not set.

----------------------------------------------------------------------

Comment By: Bernhard Reiter (ber)
Date: 2005-10-16 15:26

Message:
Logged In: YES 
user_id=113859

Hi Marc-Andre, 

do you mean that the current CVS version will return (None, None) 
always or only for special LANUGUAGE settings? 

I do not have an overview about other problems with the 
LANGUAGE variable (from gettext), but adding support 
for the proper parsing of the colons and the testing seems 
a good thing to do from my perspective. 
Getdefaultlocale() will not get called often and if additional information 
can be used from the LANGUAGE variable, this will be benefical to the 
applications. 

Anyway, 
just my 0,02 Euro-Cents. 

Bernhard R. 

----------------------------------------------------------------------

Comment By: M.-A. Lemburg (lemburg)
Date: 2005-09-26 20:23

Message:
Logged In: YES 
user_id=38388

The current CVS version returns this value:

>>> import locale
>>> locale.getdefaultlocale()
(None, None)

Given all the problems with the LANGUAGE environment variable
(which is a gettext() only thing) I'm inclined to remove
support for
it altogether.

----------------------------------------------------------------------

Comment By: Bernhard Herzog (bernhard)
Date: 2005-09-26 18:43

Message:
Logged In: YES 
user_id=2369

Another consequence of this bug is that even if
getdefaultlocale does not fail with an exception, it may
return an invalid value for the encoding.  E.g. one thuban
user had

LANGUAGE=pt_BR:pt_PT:pt

getdefaultlocale did not raise an exception, but return
"pt_pt" as the encoding because the normalized form of the
above value was pt_BR.pt_pt and the locale module assumes
that the part after the "." is the encoding.

----------------------------------------------------------------------

Comment By: mixedpuppy (mixedpuppy)
Date: 2005-03-10 22:50

Message:
Logged In: YES 
user_id=1234417

IMHO the proper behaviour is to split on the colon, then try
each one from start to finish until there is a success, or
all fail.  For example, if you just try en_DK, you will get
a failure since that is not in locale.locale_alias, but
en_GB or en_US would succeed.

----------------------------------------------------------------------

Comment By: Serge Orlov (sorlov)
Date: 2005-03-10 19:48

Message:
Logged In: YES 
user_id=1235914

The docs for getdefaultlocale state that it follows the GNU
gettext search path. OTOH gettext can return result from any
of catalogs en_DK:en_GB:en_US:en, it depends on the content
of the message. So maybe getdefaultlocale should just pick
up the first value from LANGUAGE ?

----------------------------------------------------------------------

Comment By: M.-A. Lemburg (lemburg)
Date: 2005-03-10 16:43

Message:
Logged In: YES 
user_id=38388

The URL you gave does state that LANGUAGE can take mulitple
entries separated by colons. However, I fail to see how to
choose the locale from the list of possibilities. Any ideas ?

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1158490&group_id=5470