[Python-Dev] Python in Unicode context

[M.-A. Lemburg]

[Source code encoding and string literals]

> I hope that my explanation above helps at seeing that source encoding
> and choice of string literals are not as independent as one may think.
> A choice that I surely do _not_ have is to see bugs appear in programs
> merely because I changed the source encoding.  Going from ISO 8859-1 to
> ISO 8859-15 for a Python source is probably fairly safe, because there
> is no need for switching the narrowness of strings.  Going from ISO
> 8859-1 to UTF-8 is very unsafe, and editing all literal strings from
> narrow to wide, using `u' prefixes, becomes almost unavoidable.

Indeed. As always: explicit is better than implicit :-) The small
"u" in front of the literal will tell all readers: this is Unicode
text.

We might introduce more ways to switch string literal interpretation
depending on module or interpreter process scope. However, the small
u is here to stay and it's available now, so why not use it ?

Your concerns about programs breaking because of changes to the
source encoding are valid, but not something that Python can address.
You have the same problem with normal text documents: a spell
checker might find wrong spellings of a word as a result of using
a wrong encoding, but it is not fool proof and things get worse
if you have multiple languages embedded in your program code.

As general advice for writing i18n compliant programs, I can
only suggest to keep programs written using a single source
encoding and language that appeals to the programmer and place
*all* string literals under gettext or similar tool control.

I usually write programs in ASCII or Latin-1 and use English
to write the string literals which then get mapped to user
languages as necessary by means of gettext or custom translation
logic.

-- 
Marc-Andre Lemburg
eGenix.com

Professional Python Services directly from the Source  (#1, Aug 05 2004)
 >>> Python/Zope Consulting and Support ...        http://www.egenix.com/
 >>> mxODBC.Zope.Database.Adapter ...             http://zope.egenix.com/
 >>> mxODBC, mxDateTime, mxTextTools ...        http://python.egenix.com/
________________________________________________________________________

::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! ::::