[Python-Dev] PEP 383: Non-decodable Bytes in System Character Interfaces
hrvoje.niksic at avl.com
Tue Apr 28 14:46:11 CEST 2009
Thomas Breuel wrote:
> But the biggest problem with the proposal is that it isn't needed: if
> you want to be able to turn arbitrary byte sequences into unicode
> strings and back, just set your encoding to iso8859-15. That already
> works and it doesn't require any changes.
Are you proposing to unconditionally encode file names as iso8859-15, or
to do so only when undecodeable bytes are encountered?
If you unconditionally set encoding to iso8859-15, then you are
effectively reverting to treating file names as bytes, regardless of the
locale. You're also angering a lot of European users who expect
If you switch to iso8859-15 only in the presence of undecodable UTF-8,
then you have the same round-trip problem as the PEP: both b'\xff' and
b'\xc3\xbf' will be converted to u'\u00ff' without a way to
unambiguously recover the original file name.
More information about the Python-Dev