[Python-checkins] gh-95027: Fix regrtest stdout encoding on Windows (GH-98492)
miss-islington
webhook-mailer at python.org
Fri Oct 21 10:52:23 EDT 2022
https://github.com/python/cpython/commit/b2aa28eec56d07b9c6777b02b7247cf21839de9f
commit: b2aa28eec56d07b9c6777b02b7247cf21839de9f
branch: 3.11
author: Miss Islington (bot) <31488909+miss-islington at users.noreply.github.com>
committer: miss-islington <31488909+miss-islington at users.noreply.github.com>
date: 2022-10-21T07:52:17-07:00
summary:
gh-95027: Fix regrtest stdout encoding on Windows (GH-98492)
On Windows, when the Python test suite is run with the -jN option,
the ANSI code page is now used as the encoding for the stdout
temporary file, rather than using UTF-8 which can lead to decoding
errors.
(cherry picked from commit ec1f6f5f139868dc2c1116a7c7c878c38c668d53)
Co-authored-by: Victor Stinner <vstinner at python.org>
files:
A Misc/NEWS.d/next/Tests/2022-10-20-17-49-50.gh-issue-95027.viRpJB.rst
M Lib/test/libregrtest/runtest_mp.py
diff --git a/Lib/test/libregrtest/runtest_mp.py b/Lib/test/libregrtest/runtest_mp.py
index 13649ce10b3a..c6a4dae2df86 100644
--- a/Lib/test/libregrtest/runtest_mp.py
+++ b/Lib/test/libregrtest/runtest_mp.py
@@ -21,6 +21,9 @@
from test.libregrtest.setup import setup_tests
from test.libregrtest.utils import format_duration, print_warning
+if sys.platform == 'win32':
+ import locale
+
# Display the running tests if nothing happened last N seconds
PROGRESS_UPDATE = 30.0 # seconds
@@ -259,11 +262,16 @@ def _run_process(self, test_name: str, stdout_fh: TextIO) -> int:
self.current_test_name = None
def _runtest(self, test_name: str) -> MultiprocessResult:
+ if sys.platform == 'win32':
+ # gh-95027: When stdout is not a TTY, Python uses the ANSI code
+ # page for the sys.stdout encoding. If the main process runs in a
+ # terminal, sys.stdout uses WindowsConsoleIO with UTF-8 encoding.
+ encoding = locale.getencoding()
+ else:
+ encoding = sys.stdout.encoding
# gh-94026: Write stdout+stderr to a tempfile as workaround for
# non-blocking pipes on Emscripten with NodeJS.
- with tempfile.TemporaryFile(
- 'w+', encoding=sys.stdout.encoding
- ) as stdout_fh:
+ with tempfile.TemporaryFile('w+', encoding=encoding) as stdout_fh:
# gh-93353: Check for leaked temporary files in the parent process,
# since the deletion of temporary files can happen late during
# Python finalization: too late for libregrtest.
diff --git a/Misc/NEWS.d/next/Tests/2022-10-20-17-49-50.gh-issue-95027.viRpJB.rst b/Misc/NEWS.d/next/Tests/2022-10-20-17-49-50.gh-issue-95027.viRpJB.rst
new file mode 100644
index 000000000000..8bf1a9d33573
--- /dev/null
+++ b/Misc/NEWS.d/next/Tests/2022-10-20-17-49-50.gh-issue-95027.viRpJB.rst
@@ -0,0 +1,4 @@
+On Windows, when the Python test suite is run with the ``-jN`` option, the
+ANSI code page is now used as the encoding for the stdout temporary file,
+rather than using UTF-8 which can lead to decoding errors. Patch by Victor
+Stinner.
More information about the Python-checkins
mailing list