[New-bugs-announce] [issue42261] Windows legacy I/O mode mistakenly ignores the device encoding

[Eryk Sun]

New submission from Eryk Sun <eryksun at gmail.com>:

In Python 3.8+, legacy standard I/O mode uses the process code page from GetACP instead of the correct device encoding from GetConsoleCP and GetConsoleOutputCP. For example:

    C:\>chcp 850
    Active code page: 850
    C:\>set PYTHONLEGACYWINDOWSSTDIO=1

    C:\>py -3.7 -c "import sys; print(sys.stdin.encoding)"
    cp850
    C:\>py -3.8 -c "import sys; print(sys.stdin.encoding)"
    cp1252
    C:\>py -3.9 -c "import sys; print(sys.stdin.encoding)"
    cp1252

This is based on config_init_stdio_encoding() in Python/initconfig.c, which sets config->stdio_encoding via config_get_locale_encoding(). Cannot config->stdio_encoding be set to NULL for default behavior?

Computing this ahead of time would require separate encodings config->stdin_encoding, config->stdout_encoding, and config->stderr_encoding. And _Py_device_encoding would have to be duplicated as something like config_get_device_encoding(PyConfig *config, int fd, wchar_t **device_encoding).

----------
components: IO, Interpreter Core, Windows
messages: 380329
nosy: eryksun, paul.moore, steve.dower, tim.golden, zach.ware
priority: normal
severity: normal
stage: needs patch
status: open
title: Windows legacy I/O mode mistakenly ignores the device encoding
type: behavior
versions: Python 3.10, Python 3.8, Python 3.9

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue42261>
_______________________________________