[I18n-sig] Re: [Python-Dev] Unicode debate

Fredrik Lundh Fredrik Lundh" <effbot@telia.com
Thu, 4 May 2000 10:07:45 +0200

Sin Hang Kin <kentsin@poboxes.com> wrote:
> > I don't see it as an axiom, but rather as a design decision you make =
> > keep your language simple. Along the lines of "all values are =
> > and (now) all integer values are representable with a single type. =
> > you happy with this?
> No. A character is not just a character.
> Got to google and make a search, the return result might be an example =
> mixed encoding text:
> Search engines index pages in their natural encoding, and present the =
> as is, so the search result page will contain whatever encoding mixed =
in. If
> you see JIS, ISO 8859, Hebrew, Thai, Utf-8, Big-5, GB2312, EUC, =
> you would not be very surprise. So, if you argue that a character is a
> character is a character, how would you handle such a mixed encoding =
> mess?

by converting the encoded data, character by character, into a
single known encoding, and doing the search in there?

> No one can write an automatically convertion program for such text, =
only if
> you can treated it as 8-bit bytes you can make use of it. Otherwise =
this is
> a mess.

do you really think the google engine repeats your search in every
possible encoding?  doesn't really sound like the most efficient way
to implement a search engine...

(if you still think that encodings has anything to do with the =
are characters" rule, see http://www.w3.org/TR/charmod )