[New-bugs-announce] [issue16455] sys.getfilesystemencoding() is not the locale encoding on FreeBSD and OpenSolaris when the locale is not set

[STINNER Victor]

New submission from STINNER Victor:

On FreeBSD and OpenIndiana, sys.getfilesystemencoding() is 'ascii' when the locale is not set, whereas the locale encoding is ISO-8859-1.

This inconsistency causes different issue. For example, os.fsencode(sys.argv[1]) fails if the argument is not ASCII because sys.argv are decoded from the locale encoding (by _Py_char2wchar()).

sys.getfilesystemencoding() is 'ascii' because nl_langinfo(CODESET) is used to to get the locale encoding and nl_langinfo(CODESET) announces ASCII (or an alias of this encoding).

Python should detect this case and set sys.getfilesystemencoding() to 'iso8859-1' if the locale encoding is 'iso8859-1' whereas nl_langinfo(CODESET) announces ASCII. We can for example decode b'\xe9' with mbstowcs() and check if it fails or if the result is U+00E9.

----------
components: Unicode
messages: 175401
nosy: ezio.melotti, haypo
priority: normal
severity: normal
status: open
title: sys.getfilesystemencoding() is not the locale encoding on FreeBSD and OpenSolaris when the locale is not set
versions: Python 3.4

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue16455>
_______________________________________