On Tue, Feb 2, 2021 at 8:16 PM Victor Stinner <vstinner@python.org> wrote:
I understand that encoding=locale.get_locale_encoding() would be different from encoding="locale": encoding=locale.get_locale_encoding() doesn't call os.device_encoding(), right?
Yes.
Would it be useful to add a io.get_locale_encoding(fd)->str (maybe "get_default_encoding"?) function which gives the chosen encoding from a file descriptor, similar to open(fd, encoding="locale").encoding? The os.device_encoding() call is not obvious.
I don't think it's so useful. encoding=None is 99% same to encoding=locale.getpreferedencoding(False). On Unix, os.device_encoding() just returns locale encoding. On Windows, os.device_encoding() is very unlikely used. open() uses WindowsConsoleIO for console unless PYTHONLEGACYWINDOWSSTDIO is set and encoding for it is UTF-8. And that's why I removed the detailed behavior from the PEP. It is too detailed and almost unrelated to EncodingWarning. I wrote a simple comment in this section instead. https://www.python.org/dev/peps/pep-0597/#locale-is-not-a-codec-alias
Opt-in warning ---------------
Although ``DeprecationWarning`` is suppressed by default, emitting ``DeprecationWarning`` always when ``encoding`` option is omitted would be too noisy.
The PEP is not very clear. Does "-X warn_encoding" only emits the warning, or does it also display it by default? Does it add a warning filter for EncodingWarning?
This section is not the spec. This section is the rationale for adding EncodingWarning instead of using DeprecationWarning.
As spec saying, EncodingWarning is a subclass of Warning. So it is displayed by default. But it is not emitted by default.
When -X encoding_warning (or -X warn_default_encoding) is used, the warning is emitted and shown unless the user suppresses warnings.
I understand that EncodingWarning is always displayed by default (default warning filters don't ignore it, whereas DeprecationWarning are ignored by default), but no warning is emitted by default. Ok, that makes sense. Maybe try to say it explicitly in the PEP.
This PEP doesn't have "backward compatibility" section because the PEP doesn't break any backward compatibility.
IMO it's a good thing to always have the section, just to say that you took time to think about backward compatibility ;-) The section can be empty, like just say "there is no incompatible change" ;-)
And if developers want to support Python ~3.9 and use -X warn_default_encoding on 3.10, they need to write `encoding=getattr(io, "LOCALE_ENCODING", None)`, as written in the spec.
Maybe repeat it in the Backward Compatibility section.
It's important to provide a way to prevent the warning without losing the support for old Python versions.
will do.
The main question is if it's possible to use encoding="locale" on Python 3.6-3.9 (maybe using some ugly hacks).
No.
Hum. To write code compatible with Python 3.9, I understand that encoding=None is the closest to encoding="locale".
And I understand that encoding=getattr(io, "LOCALE_ENCODING", None) is backward and forward compatible ;-)
Well, encoding=None will hopefully remain accepted with your PEP anyway for lazy developers ;-)
Yes. I don't think this warning is enabled by default in near future. So developers can just use the option to find missing `encoding="utf-8"` bug.
Oh, I'm sorry. I want to make it in 3.10.
Since it doesn't change anything by default, the warning is only displayed when you opt-in for it, IMO Python 3.10 target is reasonable.
Victor -- Night gathers, and now my watch begins. It shall not end until my death.
-- Inada Naoki <songofacandy@gmail.com>