[New-bugs-announce] [issue13333] utf-7 inconsistent with surrogates
Antoine Pitrou
report at bugs.python.org
Thu Nov 3 13:13:47 CET 2011
New submission from Antoine Pitrou <pitrou at free.fr>:
The utf-7 codec happily encodes lone surrogates, but it won't decode them:
>>> "\ud801".encode("utf-7")
b'+2AE-'
>>> "\ud801\ud801".encode("utf-7")
b'+2AHYAQ-'
>>> "\ud801".encode("utf-7").decode("utf-7")
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "/home/antoine/cpython/default/Lib/encodings/utf_7.py", line 12, in decode
return codecs.utf_7_decode(input, errors, True)
UnicodeDecodeError: 'utf7' codec can't decode bytes in position 0-4: second surrogate missing at end of shift sequence
>>> "\ud801\ud801".encode("utf-7").decode("utf-7")
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "/home/antoine/cpython/default/Lib/encodings/utf_7.py", line 12, in decode
return codecs.utf_7_decode(input, errors, True)
UnicodeDecodeError: 'utf7' codec can't decode bytes in position 0-6: second surrogate missing
I don't know which behaviour is better but round-tripping is certainly a desirable property of any codec.
----------
components: Interpreter Core, Unicode
messages: 146919
nosy: ezio.melotti, loewis, pitrou
priority: normal
severity: normal
status: open
title: utf-7 inconsistent with surrogates
type: behavior
versions: Python 2.7, Python 3.2, Python 3.3
_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue13333>
_______________________________________
More information about the New-bugs-announce
mailing list