[issue11574] TextIOWrapper: Unicode Fallback Encoding on Python 3.3

[Martin v. Löwis]

Martin v. Löwis <martin at v.loewis.de> added the comment:

> One might say, "ok, this is a bug in distutils/distribute, it should
> explicitly specify UTF-8 encoding when writing egg-info." But if this
> is a sensible thing for distutils/distribute to do, regardless of
> user locale, why would it not be equally sensible for Python itself
> to have the default output encoding always be UTF-8 (with the ability
> for a developer who wants to support arbitrary user locale to
> explicitly do so)?

The file encoding is part of the file format. Just as Python can't know
what the file format is (else it could allow writing, say, dictionaries
to a file), it can't know what the file encoding is, either - there is
a need to guess. distutils *does* know the format, so it's clearly a
bug in distutils and not in Python.

The Zen says "In the face of ambiguity, refuse the temptation to guess."
>From that point of view, Python should just refuse to open files in text
mode with no encoding specified. However, it also says "Although
practicality beats purity.", which brings us back to guessing.

Guessing the "best" file encoding is really tricky, and Python has
chosen to use the locale's encoding. That can't be changed anymore
(except perhaps by PEP) since it would be an incompatible change.

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue11574>
_______________________________________