[Python-Dev] PEP 538 (review round 2): Coercing the legacy C locale to a UTF-8 based locale

[INADA Naoki]

Now I approve the PEP 538.

It's side-effect (just set LC_CTYPE envvar) seems simple enough and
moderate enough.

Locale coercion will save people from unwanted mojibake (escaped string)
and locale warning will navigate people to configure locale properly.

And there are configure options and envvar option to disable it for people
who want to continue to use C locale explicitly.

Congrats, Nick!

On Sat, May 27, 2017 at 4:19 PM, Nick Coghlan <ncoghlan at gmail.com> wrote:
> On 24 May 2017 at 02:34, Nick Coghlan <ncoghlan at gmail.com> wrote:
>> On 23 May 2017 at 18:38, INADA Naoki <songofacandy at gmail.com> wrote:
>>> Would you add example demonstrates how coercing LANG helps people?
>>
>> I'm honestly not sure it does - I think it's an assumption I added to
>> the PEP early on, and never actually tested. Looking at it more
>> closely now, all of the interpreter level checks are specifically for
>> LC_CTYPE, and experimenting with "LANG=C LC_CTYPE=C.UTF-8" indicates
>> that coercing only LC_CTYPE is still enough to fix the GNU readline
>> encoding compatibility problem covered in
>> https://www.python.org/dev/peps/pep-0538/#considering-locale-coercion-independently-of-utf-8-mode
>>
>> So I'll take another pass through the implementation this weekend, and
>> simplify it to only set LC_CTYPE regardless of whether it's using
>> C.UTF-8, C.utf8, or UTF-8 as the target locale. Assuming that doesn't
>> uncover any hidden problems with the idea, I'll then update the PEP to
>> match.
>
> I've now gone through this, and as far as I can tell, setting only
> LC_CTYPE is sufficient to handle all the scenarios that the PEP aims
> to address, and has fewer potential side effects than setting both
> LC_CTYPE and LANG.
>
> Accordingly, I've updated both the PEP and the implementation to only
> set LC_CTYPE and leave LANG alone:
>
> * PEP: https://github.com/python/peps/commit/12cecb05489e74a36a11c17e8d0b1e36e3768bda
> * Implementation:
> https://github.com/python/cpython/pull/659/commits/939ba0a77d4b52a04315c129f9db89b90c0532cd
>
> Regards,
> Nick.
>
> --
> Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia