[issue9630] Reencode filenames when setting the filesystem encoding

[STINNER Victor]

STINNER Victor <victor.stinner at haypocalc.com> added the comment:

Forget my previous message, I forgot important points.

> So the only reason why you have to go through
> all those hoops is to
>
> * allow the complete set of Python supported encoding
>   names for the PYTHONFSENCODING
>
> * make sure that the Py_FilesystemDefaultEncoding is set
>   to the actual name of the codec as used by the system

Not only. As I wrote in my first message (msg114191), there are two
other good reasons to keep the current code but redecode filenames:

 * Encoding aliases: locale encoding is not always written as the
   official Python encoding name. Eg. utf8 vs UTF-8, iso8859-1 vs
   latin_1, etc. We have to be able to load Lib/encodings/aliases.py to
   to get the Python codec.

 * Codecs implemented in Python: only ascii, latin1, utf8 and mbcs
   codecs are builtin. All other encodings are implemented in Python. If
   your filesystem encoding is ShiftJIS, you have to load
   Lib/encodings/shift_jis.py to load the codec.

For these two reasons, we have to import Python modules before being
able to set the filesystem encoding. So we have to redecode filenames
after setting the filesystem encodings.

> the redecoding of the filenames is fragile

We can setup a buildbot installed in a non-ascii path. Antoine had such
buildbot, which already helped to find many bugs related to non-ascii paths.

--

We can choose to only support ascii, latin1, utf8 and mbcs for the
filesystem encoding, but users will complain that we break compatibility
with old systems. Python3 already "breaks" the language, I don't think
that it is a good idea to choose to become incompatible with old systems
just to simplify (too much) the code.

--

Another solution would be to unload all modules, clear all caches,
delete all code objects, etc. after setting the filesystem encoding. But
I think that it is inefficient and nobody wants a slower Python startup.

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue9630>
_______________________________________