[issue9542] Create PyUnicode_FSDecoder() function

[STINNER Victor]

New submission from STINNER Victor <victor.stinner at haypocalc.com>:

For my work on #9425 (Rewrite import machinery to work with unicode paths), I need a PyArg_Parse converter converting bytes and str to str. PyUnicode_FSConverter() is the opposite because it encodes str to bytes.

To handle (input) filenames in a function, we have 3 choices:

 1/ use bytes: that's the current choice for most Python functions. It gives full unicode support for POSIX OSes (FS using a bytes API), but it is not enough for Windows (Windows uses mbcs encoding which is a very small subset of Unicode)
 2/ use str with the PEP 383 (surrogateescape): it begins to be used in Python 3.1, and more seriously in Python 3.2. It offers full unicode support on all OSes (POSIX and Windows)
 3/ use the native type for each OS (bytes on POSIX, str on Windows): I dislike this solution because it implies code duplication

PyUnicode_FSConverter() is the converter for solution (1). PyUnicode_FSDecoder() will be the converter for the solution (2).

----------
components: Interpreter Core, Unicode
files: PyUnicode_FSDecoder.patch
keywords: patch
messages: 113352
nosy: haypo, lemburg, loewis
priority: normal
severity: normal
status: open
title: Create PyUnicode_FSDecoder() function
versions: Python 3.2
Added file: http://bugs.python.org/file18447/PyUnicode_FSDecoder.patch

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue9542>
_______________________________________