[New-bugs-announce] [issue8969] Windows: use (mbcs in) strict mode to encode/decode filenames, and enable os.fsencode()

[STINNER Victor]

New submission from STINNER Victor <victor.stinner at haypocalc.com>:

mbcs encoding doesn't support surrogateescape (see #850997), and mbcs should only be used in strict mode to encode/decode filenames.

os.fsencode() should also be enabled on Windows. First I tried to disable this function on Windows to avoid the evil mbcs encoding, but mbcs encoding *is* used by some modules written in C (functions using PyUnicode_FSConverter(): encode the filename to bytes with mbcs encoding on Windows). Eg. _ssl module use PyUnicode_FSConverter() to get filenames because the underlying library, OpenSSL, requires bytes for the filenames (C type: char*). Enable os.fsencode() on Windows helps some tests (eg. fix test_ssl).

Use "strict" error handler, instead of "surrogateescape", to encode/decode filenames with mbcs encoding, does nothing yet because mbcs codec ignore the errors argument. These changes prepare the work on mbcs codec: see #850997.

Note: os.fsencode() was introduced by #8514.

----------
components: Interpreter Core, Library (Lib), Unicode, Windows
files: fsencode_mbcs.patch
keywords: patch
messages: 107510
nosy: haypo
priority: normal
severity: normal
status: open
title: Windows: use (mbcs in) strict mode to encode/decode filenames, and enable os.fsencode()
versions: Python 3.2
Added file: http://bugs.python.org/file17616/fsencode_mbcs.patch

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue8969>
_______________________________________