[New-bugs-announce] [issue3995] iso-xxx/cp1252 inconsistencies in Python 2.* not in 3.*

[Jean-Michel Fauth]

New submission from Jean-Michel Fauth <wxjmfauth at gmail.com>:

XP SP2 fr_CH cp1252

I have always found, there are some inconsistencies in the Python <=2.5
serie regarding the char endodings, especially the iso-8859-1, cp1252,
iso-8859-15 encodings.

I do not know if this must be considered as a bug or as a feature.
Python is quite friendly with these encodings. It may not be a problem
for a daily work, it is more acute when one wish to teach the
chararacter encodings.

char "œ": "code point" 156 in cp1252
char "€": "code point" 128 in cp1252

Python 2.5.2

>>> unicode('œ', 'cp1252')
u'\u0153'
>>> unicode('€', 'cp1252')
u'\u20ac'
>>> unicode('œ', 'iso-8859-15')
u'\x9c'
>>> unicode('€', 'iso-8859-15')
u'\x80'
>>> unicode('€', 'iso-8859-1') #***
u'\x80'
>>> unicode('œ', 'iso-8859-1') #***
u'\x9c'
>>> #*** should raise an error since œ and €
>>> #are not existing in an iso-8859-1 table.
>>> 

It looks like iso-8859-1 behaves as iso-8859-15 (typo somewhere?)

Python 3.0 rc1 does the job correctly and notices the difference

>>> bytes('œ', 'cp1252')
b'\x9c'
>>> bytes('€', 'cp1252')
b'\x80'
>>> bytes('œ', 'iso-8859-15')
b'\xbd'
>>> bytes('€', 'iso-8859-15')
b'\xa4'
>>> bytes('œ', 'iso-8859-1')
Traceback (most recent call last):
  File "<pyshell#5>", line 1, in <module>
    bytes('œ', 'iso-8859-1')
UnicodeEncodeError: 'latin-1' codec can't encode character '\u0153' in
position 0: ordinal not in range(256)
>>> bytes('€', 'iso-8859-1')
Traceback (most recent call last):
  File "<pyshell#6>", line 1, in <module>
    bytes('€', 'iso-8859-1')
UnicodeEncodeError: 'latin-1' codec can't encode character '\u20ac' in
position 0: ordinal not in range(256)
>>> # these errors are expected
>>> 

Python 2.6**

The latest version is not installed. If I recall correcly, 2.6b* are
presenting the same issue as in 2.5.2 .

----------
messages: 74017
nosy: jmfauth
severity: normal
status: open
title: iso-xxx/cp1252  inconsistencies in Python 2.* not in 3.*
versions: Python 2.5

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue3995>
_______________________________________