[Python-Dev] Issue #8610: Set default file system encoding to ascii on error?

[Victor Stinner]

Python 3.0 introduced PyUnicode_DecodeFSDefault() and 
PyUnicode_DecodeFSDefaultAndSize() functions. These functions fallback to 
UTF-8 if getting the file system encoding failed or the encoding is unknown 
(there is not nl_langinfo(CODESET) function).

UTF-8 is not a good choice for the fallback because it's incompatible with 
other encodings like Latin1. I would like to fallback to ASCII on error which 
is compatible with all encodings (thanks to surrogateescape).

I would like to ensure that sys.getfilesystemencoding() result cannot be None, 
because Python 3.2 uses it on Unix to decode os.environb and to encode 
filenames in subprocess. I can implement a fallback for os.environb and 
subprocess (and other functions calling sys.getfilesystemencoding()), but I 
prefer to have a reliable sys.getfilesystemencoding() function.

This change doesn't concern Windows and Mac OS X because the encoding is 
hardcoded (mbcs, utf-8). On Unix, I don't know in which case nl_langinfo() can 
fail. Empty environment is not an error: nl_langinfo(CODESET) returns "ascii". 
I think that few (or no) user would notice this change.

-- 
Victor Stinner
http://www.haypocalc.com/