[New-bugs-announce] [issue34618] Encoding error running in subprocess with captured output

[Jeremy Kloth]

New submission from Jeremy Kloth <jeremy.kloth+python-tracker at gmail.com>:

When running Python via subprocess with captured output an encoding error occurs attempting to output a Unicode filename.  The same does not happen when just using spawnl().

Python 3.6.5 (v3.6.5:f59c0932b4, Mar 28 2018, 17:00:18) [MSC v.1900 64 bit (AMD64)] on win32
Type "help", "copyright", "credits" or "license" for more information.
>>> import os, sys, subprocess
>>> sys.stdout.encoding, sys.stdout.errors
('utf-8', 'surrogateescape')
>>> args = ['-u', '-c', "print('taqdir\\\u0634\u0645\u0627\u0631.py')"]
>>> os.spawnl(os.P_WAIT, sys.executable, '"%s"' % sys.executable, *args)
taqdir\شمار.py
0
>>> subprocess.run([sys.executable, *args], stdout=subprocess.PIPE, stderr=subprocess.STDOUT)
CompletedProcess(args=['C:\\Program Files (x86)\\Microsoft Visual Studio\\Shared\\Python36_64\\python.exe', '-u', '-c', "print('taqdir\\شمار.py')"], returncode=1, stdout=b'Traceback (most recent call last):\r\n  File "<string>", line 1, in <module>\r\n  File "C:\\Program Files (x86)\\Microsoft Visual Studio\\Shared\\Python36_64\\lib\\encodings\\cp1252.py", line 19, in encode\r\n    return codecs.charmap_encode(input,self.errors,encoding_table)[0]\r\nUnicodeEncodeError: \'charmap\' codec can\'t encode characters in position 7-10: character maps to <undefined>\r\n')

----------
components: Interpreter Core, Library (Lib), Unicode, Windows
messages: 324904
nosy: ezio.melotti, jkloth, paul.moore, steve.dower, tim.golden, vstinner, zach.ware
priority: normal
severity: normal
status: open
title: Encoding error running in subprocess with captured output
versions: Python 3.6, Python 3.7, Python 3.8

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue34618>
_______________________________________