[issue1552880] [Python2] Use utf-8 in the import machinery on Windows to support unicode paths
report at bugs.python.org
Sat Sep 4 03:00:41 CEST 2010
STINNER Victor <victor.stinner at haypocalc.com> added the comment:
Oh, I didn't see that the issue was specific to Python2. I updated the issue's title. If I understood correctly, the issue is also specific to Windows.
Do you know if your patch changes the public API? (break the compatibility)
FYI about Python3:
> That's an inventive way of breaking the unicode standard :)
It is described in the PEP 383 and it does solve a real and common issue: store a filename that cannot be decoded with the filesystem encoding. The operation is reversible. In Python 3.2, there are os.fsdecode() and os.fsencode() functions. On UNIX/BSD, os.encode(os.fsdecode(x)) is x, if x is a bytes object.
The PEP 383 introduces the surrogateescape error handler which does create surrogates on decode, and convert back surrogates to bytes on encode.
> Anyway, why would you worry about that? My patch doesn't use
> "surrogateescape" so there is no problem.
In Python3, filenames are stored as unicode. On UNIX/BSD, if a filename cannot be decode, it is encoded with surrogates. To get a full unicode support in Python3, you have to support surrogates.
title: Unicode Imports -> [Python2] Use utf-8 in the import machinery on Windows to support unicode paths
Python tracker <report at bugs.python.org>
More information about the Python-bugs-list