[issue9988] test_warnings fails with PYTHONFSENCODING=latin-1 on UNIX/BSD

[STINNER Victor]

New submission from STINNER Victor <victor.stinner at haypocalc.com>:

$ PYTHONFSENCODING=latin-1 ./python Lib/test/test_warnings.py 
...
======================================================================
FAIL: test_nonascii (__main__.CEnvironmentVariableTests)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "Lib/test/test_warnings.py", line 731, in test_nonascii
    "['ignore:DeprecaciónWarning']".encode('utf-8'))
AssertionError: b"['ignore:Deprecaci\\udcf3nWarning']" != b"['ignore:Deprecaci\xc3\xb3nWarning']"

======================================================================
FAIL: test_nonascii (__main__.PyEnvironmentVariableTests)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "Lib/test/test_warnings.py", line 731, in test_nonascii
    "['ignore:DeprecaciónWarning']".encode('utf-8'))
AssertionError: b"['ignore:Deprecaci\\udcf3nWarning']" != b"['ignore:Deprecaci\xc3\xb3nWarning']"

----------------------------------------------------------------------

The problem is that subprocess encodes PYTHONWARNINGS environment variable value with the filesystem encoding, whereas Py_main() decodes the variable value with the locale encoding.

History of how the variable is read in py3k:
 - #7301: r79880 creates this variable, use mbstowcs() and PySys_AddWarnOption()
 - #7301: r80066 uses setlocale(LC_ALL, ""), and replaces mbstowcs() by _Py_char2wchar() (to support surrogates)
 - #8589: r81358 creates PySys_AddWarnOptionUnicode() and replaces _Py_char2wchar() by PyUnicode_DecodeFSDefault()
 - #8589: r84694 replaces PyUnicode_DecodeFSDefault() by _Py_char2wchar() + PyUnicode_FromWideChar() "because the PyCodec machinery is not ready yet"
 - #8589: r84731 uses PyUnicode_FromString() (utf-8) on Mac OS X

----------
components: Tests, Unicode
messages: 117631
nosy: haypo
priority: normal
severity: normal
status: open
title: test_warnings fails with PYTHONFSENCODING=latin-1 on UNIX/BSD
versions: Python 3.2

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue9988>
_______________________________________