[Python-bugs-list] [ python-Bugs-544609 ] cant encode unicode string to iso-8859-2

noreply@sourceforge.net noreply@sourceforge.net
Tue, 16 Apr 2002 08:25:18 -0700


Bugs item #544609, was opened at 2002-04-16 13:10
You can respond by visiting: 
http://sourceforge.net/tracker/?func=detail&atid=105470&aid=544609&group_id=5470

Category: Unicode
Group: Python 2.2.1 candidate
>Status: Closed
>Resolution: Invalid
Priority: 5
Submitted By: Pavel Stehule (okbob)
Assigned to: M.-A. Lemburg (lemburg)
Summary: cant encode unicode string to iso-8859-2

Initial Comment:
>>> import sys
>>> sys.getdefaultencoding()
'ISO8859-2'
>>> s = u"Příliš žluoučký kůň se napil žluté vody"
>>> s.encode("iso-8859-2")
Traceback (most recent call last):
  File "<stdin>", line 1, in ?
  
File "/usr/local/lib/python2.2/encodings/iso8859_2.py",
 line 18, in encode
    return codecs.charmap_encode
(input,errors,encoding_map)
UnicodeError: charmap encoding error: character maps 
to <undefined>

But I used only chars from iso-8859-2. But If didn't 
use czech chars, encoding is successfull

----------------------------------------------------------------------

>Comment By: Martin v. Löwis (loewis)
Date: 2002-04-16 17:25

Message:
Logged In: YES 
user_id=21627

You are confused about the meaning of Unicode literals. Even
though it might appear to you that the literal denotes
Latin-2 characters, Python has no way of knowing that the
bytes between the u" and the " are meant as Latin-2. So it
always takes them as Latin-1. This will be fixed with PEP 263.

However, once you constructed the string s, it is not
surprising that you cannot encode it: You did *not* use
characters from Latin-2, you've used characters from
Latin-1; some of those cannot be encoded in Latin-2, hence
the UnicodeError.

In Python 2.2, the proper solution is to avoid Unicode
literals, and use the unicode function to construct Unicode
strings:

s = unicode("your text", "latin-2")

So this is, strictly speaking, not a bug.

----------------------------------------------------------------------

You can respond by visiting: 
http://sourceforge.net/tracker/?func=detail&atid=105470&aid=544609&group_id=5470