The fact that Python does not use UTF-8 as the default encoding when
opening text files is an obstacle for many Windows users, especially
beginners in programming.
If you search for UnicodeDecodeError, you will see that many Windows
users have encountered the problem.
This list is only part of many search results.
* https://qiita.com/Yuu94/items/9ffdfcb2c26d6b33792e
* https://www.mikan-partners.com/archives/3212
* https://teratail.com/questions/268749
* https://github.com/neovim/pynvim/issues/443
* https://www.coder.work/article/1284080
* https://teratail.com/questions/271375
* https://qiita.com/shiroutosan/items/51358b24b0c3defc0f58
* https://github.com/jsvine/pdfplumber/issues/304
* https://ja.stackoverflow.com/questions/69281/73612
* https://trend-tracer.com/pip-error/
Looking at the errors, the following are the most common cases.
* UnicodeDecodeError is raised when trying to open a text file written
in UTF-8, such as JSON.
* UnicodeEncodeError is raised when trying to save text data retrieved
from the web, etc.
* User run `pip install` and `setup.py` reads README.md or LICENSE
file written in UTF-8 without `encoding="UTF-8"`
Users can use UTF-8 mode to solve these problems.
I wrote a section for UTF-8 mode in the "3. Using Python on Windows" document.
https://docs.python.org/3/using/windows.html#utf-8-mode
However, UTF-8 mode is still not very well known. How can we make
UTF-8 mode more user-friendly?
Right now, UTF-8 mode can be enabled using the `-Xutf8` option or the
`PYTHONUTF8` environment variable. This is a hurdle for beginners. In
particular, Jupyter users may not use the command line at all.
Is it possible to enable UTF-8 mode in a configuration file like `pyvenv.cfg`?
* User can enable UTF-8 mode per-install, and per-venv.
* But difficult to write the setting file when Python is installed for
system (not for user), or Windows Store Python
* User can still enable UTF-8 mode in venv. But many beginners don't
need venv.
Is it possible to make it easier to configure?
* Put a checkbox in the installer?
* Provide a small tool to allow configuration after installation?
* python3 -m utf8mode enable|disable?
* Accessible only for CLI user
* Add "Enable UTF-8 mode" and "Disable UTF-8 mode" to Start menu?
Any ideas are welcome.
--
Inada Naoki