[issue11574] TextIOWrapper: Unicode Fallback Encoding on Python 3.3

Carl Meyer report at bugs.python.org
Wed Dec 14 21:45:51 CET 2011

Carl Meyer <carl at dirtcircle.com> added the comment:

Here's an example real-world case where the only solution I could find was to simply avoid non-ASCII characters entirely (which is obviously not a real solution): https://github.com/pypa/virtualenv/issues/201#issuecomment-3145690

distutils/distribute require long_description to be a string, not bytes (so it can rfc822-escape it, and use string methods to do so), but does not explicitly set an output encoding when it writes egg-info. This means that a developer either has the choice to a) break installation of their package on any system with an ASCII default locale, or b) not use any non-ASCII characters in long_description.

One might say, "ok, this is a bug in distutils/distribute, it should explicitly specify UTF-8 encoding when writing egg-info." But if this is a sensible thing for distutils/distribute to do, regardless of user locale, why would it not be equally sensible for Python itself to have the default output encoding always be UTF-8 (with the ability for a developer who wants to support arbitrary user locale to explicitly do so)?

nosy: +carljm

Python tracker <report at bugs.python.org>

More information about the Python-bugs-list mailing list