Do we need an option to leave the LC_CTYPE locale unchanged?
Hi,
I'm working on my PEP 587 "Python Initialization Configuration": https://www.python.org/dev/peps/pep-0587/
I just proposed to add a new "configure_locale" configuration parameter to leave the LC_CTYPE locale unchanged: https://bugs.python.org/issue36945
Right now, Python always calls setlocale(LC_CTYPE, "") on all platforms to set the LC_CTYPE locale to the user preferred locale.
Is it a problem when Python is embedded in an application? Or am I trying to fix an artificial issue? :-)
By the way, Nick Coghlan and me decided to disable LC_CTYPE locale coercion (PEP 538) and UTF-8 Mode (PEP 540) by default in Python 3.8. It should reduce the risk of mojibake.
Victor
Night gathers, and now my watch begins. It shall not end until my death.
On 16May2019 1632, Victor Stinner wrote:
Hi,
I'm working on my PEP 587 "Python Initialization Configuration": https://www.python.org/dev/peps/pep-0587/
I just proposed to add a new "configure_locale" configuration parameter to leave the LC_CTYPE locale unchanged: https://bugs.python.org/issue36945
Right now, Python always calls setlocale(LC_CTYPE, "") on all platforms to set the LC_CTYPE locale to the user preferred locale.
Is it a problem when Python is embedded in an application? Or am I trying to fix an artificial issue? :-)
I posted on the bug, but the answer to the question in the subject line is "no" :)
Embedding Python shouldn't change any global settings in your C runtime. We should be good citizens and work with whatever is there, and if it causes issues for the host application, then we can say "you should probably change this global setting".
So we don't need an option. We just have to move our setlocale() call from Py_Initialize() into Py_Main()
Cheers, Steve
Hi Steve,
Le ven. 17 mai 2019 à 18:42, Steve Dower <steve.dower@python.org> a écrit :
Embedding Python shouldn't change any global settings in your C runtime. We should be good citizens and work with whatever is there, and if it causes issues for the host application, then we can say "you should probably change this global setting".
Ok, I added an option to decide if the locale is configured or left unchanged.
To be clear, Py_Initialize() in Python 3 always started with setting LC_CTYPE to the user preferred locale. The new option to disable this behavior is a new feature. It wasn't possible to leave the locale unchanged previously.
So we don't need an option. We just have to move our setlocale() call from Py_Initialize() into Py_Main()
Well, the implementation is super complex. It's not easy to move setlocale() call from Py_Initialize() into Py_Main(). I'm not sure that it's doable.
You need the locale to be configured properly to be able to call Py_Initialize(). Too many things rely on the locale. If you change the locale too late, you introduce mojibake. For example, you might no longer to be able to open file if you read the filename before the locale changed.
That's why Nick Coghlan asked me to even introduce a "pre-initialization" phase to really ensure that the locale is configured *before* Py_Initialize().
An option is just a very simple practical solution to a super complex problem :-)
And people are using Python is so many different ways, I prefer to leave the door open to let people decide what Python do ;-)
Victor
Night gathers, and now my watch begins. It shall not end until my death.
On 17May2019 1929, Victor Stinner wrote:
Le ven. 17 mai 2019 à 18:42, Steve Dower <steve.dower@python.org> a écrit :
So we don't need an option. We just have to move our setlocale() call from Py_Initialize() into Py_Main()
Well, the implementation is super complex. It's not easy to move setlocale() call from Py_Initialize() into Py_Main(). I'm not sure that it's doable.
By Py_Main *calls* Py_Initialize (via the new APIs), so it can do whatever it likes before Py_Initialize is called. We'd be calling setlocale *earlier*, not later.
Cheers, Steve
participants (2)
-
Steve Dower -
Victor Stinner