[Python-ideas] PEP 540: Add a new UTF-8 mode

Steven D'Aprano steve at pearwood.info
Thu Jan 5 18:35:09 EST 2017

On Thu, Jan 05, 2017 at 04:38:22PM +0100, Victor Stinner wrote:

> Python 3 promotes Unicode everywhere including filenames. A solution to
> support filenames not decodable from the locale encoding was found: the
> ``surrogateescape`` error handler (`PEP 393
> <https://www.python.org/dev/peps/pep-0393/>`_), store undecodable bytes
> as surrogate characters.

PEP 393 is the Flexible String Respresentation.

I think you want PEP 383, Non-decodable Bytes in System Character 


> The problem is that operating system data like filenames are decoded
> using the ``surrogateescape`` error handler (PEP 393).



More information about the Python-ideas mailing list