On Sat, Jan 30, 2021 at 3:45 PM Christopher Barker
On Thu, Jan 28, 2021 at 4:25 PM Inada Naoki
wrote: The "real" solution is to change the defaults not to use the system encoding at all -- which, of course, we are moving towards with PEP 597. So first a plug to do that as fast as possible! I myself would love to see PEP 597 implemented tomorrow -- for all supported versions of Python.
Note that PEP 597 doesn't change the default encoding. It just adds an option to emit a warning when the default encoding is used.
I know -- and THAT could be done soon, yes?
Sorry for the delay. I want to do it in Python 3.10, but I am not sure the PEP is accepted. I updated the PEP today and working on reference implementation now.
I think it might take about 10 years to change it.
I hope it's not that long -- having code that runs differently in different environments is not good ...
I agree. But backward compatibility is important too.
Many codes are written by other people. It cause UnicodeDecodeError on Windows. And UTF-8 mode rescues it.
exactly. But the trick is that UTF-* mode is in control of the end user / installer of Python, not the writer of the code.
Yes. But the writer of the code can specify `encoding="utf-8"`. PEP 597 helps it. Additionally, I am proposing per environment option. If code owner distributes the application by embeddable Python or venv, they can use UTF-8 mode too.
Please don't discuss PEP 597 in this thread. Let's focus on UTF-8 mode. They are different approaches and they are not mutually exclusive.
Sure, but they are related. But I"ll try to find the right thread for PEP 597
The thread for the PEP is https://discuss.python.org/t/pep-597-raise-a-warning-when-encoding-is-omitte...
Imagine someone runs some code in Jupyter, and it's fine, and then they run it in plain Python, on the same machine, and it breaks -- ouch!
You are right. UTF-8 mode must be accessible for both of Jupyter on conda Python and Python installed by official installer. If UTF-8 mode is accessible enough, user can fix it by enabling UTF-8 mode.
Sure -- but these days folks may have multiple environments and multiple ways to run code (Jupyter, IDEs), so it's way too easy to have UTF-8 mode on in some but not others -- all on the same machine.
If the user don't have legacy application, they can set PYTHONUTF8 as user environment variable. Then all environments are working on UTF-8 mode.
I'm not a Windows user (much), but users of my library are, and my students are, and I'm having a hard time figuring out what will make this work for them.
In the case of my students, I can encourage UTF-8 mode for all installations.
Yes, I think UTF-8 mode will help teachers and students. Maybe, WSL2.will be another option.
In the case of my library users -- it's harder, but I can do the same to some extent -- I do currently suggest a conda environment for my code -- so yes, making it easier to turn it on in an environment would be good.
If PEP 597 is accepted, you can find all code omitting `encoding="utf-8"`. Your library users can run it without UTF-8 mode.
It is powerful/flexible for power users. But not for beginners. Imagine users execute Jupyter from the start menu.
* Command-line `-Xutf8` or `set PYTHONUTF8=1` is not accessible. * User environment variable is not accessible too, and it may affect other Python installations.
which is actually what I like about environment variables -- it could apply to all Python installations on the system -- which would be a good thing!
It will be great for many users. But it will be not good for some users using legacy Python applications. So it is difficult to recommend UTF-8 mode to everyone.
Where would Python look for a "configuration file like `pyvenv.cfg`" ?
I am not a Windows expert so I am not sure. But I think it should be the same directory where `python.exe` is in.
Back to my idea above -- any way to have that be a pip (and conda) installable package? So it could be in a requirements file?
I have no idea.
--
Inada Naoki