[issue27179] subprocess uses wrong encoding on Windows

[Eryk Sun]

Eryk Sun added the comment:

> so if we default to UTF-8 it will be even worse than 
> defaulting to ANSI because there aren't many programs 
> on Windows which would use UTF-8

I didn't say subprocess should default to UTF-8. What I wish is for Python to default to using UTF-8 for its own pipe and disk file I/O. The old behavior could be selected by setting some hypothetical environment variable, such as PYTHONIOUSELOCALE.

If subprocess defaults to the console's current codepage (when available), it would be nice to have a way to conveniently select the OEM or ANSI codepage. The codecs module could define string constants based on GetOEMCP() and GetACP(), such as codecs.CP_OEMCP (e.g. 'cp437') and codecs.CP_ACP (e.g. 'cp1252'). subprocess could import these constants on Windows.

> for example "cmd /u /c chcp.exe" will return result 
> in UTF-16 because such program doesn't exist and cmd's 
> error message will be outputted.

Yes, that's a problem. The cases where we want UTF-16 from the shell should be handled specially instead of enabled generally.

> Also if user have set.exe in %System32% then 
> "cmd /u /c set" won't be in UTF-16 because it will 
> execute that program.

This one's not actually a problem because cmd defaults to parsing `set` as its internal command. An external `set` requires quotes (e.g. `"set"` to look for set, set.com, set.exe, ...).

> you want to get GetConsoleOutputCP before program's
> exit and not at start (say with CREATE_SUSPENDED) 
> as it might have changed it somewhere in middle of
> program's execution.

You'd have to pass both the process ID and the thread ID to have the monitor call OpenProcess to get a waitable handle and OpenThread to call ResumeThread. It can be done, but it's not something I'd consider doing in practice. It's too fragile and not worth the trouble for something that's rarely required.

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue27179>
_______________________________________