[issue9630] Reencode filenames when setting the filesystem encoding

STINNER Victor report at bugs.python.org
Fri Sep 24 13:52:59 CEST 2010

STINNER Victor <victor.stinner at haypocalc.com> added the comment:

> Why is this needed ?

Short answer: to support filesystem encoding different than utf-8. See #8611 for a longer explanation.


$ pwd
$ PYTHONFSENCODING=ascii ./python test_fs_encoding.py 
Fatal Python error: Py_Initialize: can't initialize sys standard streams
UnicodeEncodeError: 'ascii' codec can't encode character '\xe9' in position 20: ordinal not in range(128)

My patch fixes this specific case and prepare the work for the complete fix (support different *locale* encodings, see #8611 and #9425).


Longer answer: Py_FilesystemDefaultEncoding is changed too late. Some modules are already loaded, sys.executable is already set, etc. Py_FilesystemDefaultEncoding is changed but modules filenames are decoded with utf-8 and should be "redecoded".

It is not possible to set Py_FilesystemDefaultEncoding before loading the first module. initfsencoding() loads codecs and encodings modules to check the codec name. sys.executable is also set before initfsencoding().

Read my other messages of this issue to get other reasons why the patch is needed. I explained other possibilities (but they don't work).


Python tracker <report at bugs.python.org>

More information about the Python-bugs-list mailing list