[Python-Dev] New _Py_InitializeFromConfig() function (PEP 432)

Victor Stinner vstinner at redhat.com
Tue Jul 31 18:14:28 EDT 2018


Hi,

I finished my work on the _PyCoreConfig structure: it's a C structure
in Include/pystate.h which has many fields used to configure Python
initialization. In Python 3.6 and older, these parameters were scatted
around the code, and it was hard to get an exhaustive list of it.

This work is linked to the Nick Coghlan's PEP 432 "Restructuring the
CPython startup sequence":
https://www.python.org/dev/peps/pep-0432/

Right now, the new API is still private. Nick Coghlan splitted the
initialization in two parts: "core" and "main". I'm not sure that this
split is needed. We should see what to do, but it would be nice to
make the _PyCoreConfig API public! IMHO it's way better than the old
way to configuration Python initialization.

--

It is now possible to only use _PyCoreConfig to initialize Python: it
overrides old ways to configure Python like environment variables (ex:
PYTHONPATH), global configuration variables (ex: Py_BytesWarningFlag)
and C functions (ex: Py_SetProgramName()).

I added tests to test_embed on the different ways to configure Python
initialization:

* environment variables (ex: PYTHONPATH)
* global configuration variables (ex: Py_BytesWarningFlag) and C
functions (ex: Py_SetProgramName())
* _PyCoreConfig

I found and fixed many issues when writing these tests :-)

Reading the current configuration, _PyCoreConfig_Read(), no longer
changes the configuration. Now the code to read the configuration and
the code to apply the configuration is properly separated.

The work is not fully complete, there are a few remaining corner cases
and some parameters (ex: Py_FrozenFlag) which cannot be set by
_PyCoreConfig yet. My latest issue used to work on this API:

https://bugs.python.org/issue34170

I had to refactor a lot of code to implement all of that.

--

The problem is that Python 3.7 got the half-baked implementation, and
it caused issues:

* Calling Py_Main() after Py_Initialize() fails with a fatal error on
Python 3.7.0
  https://bugs.python.org/issue34008
* PYTHONOPTIMIZE environment variable is ignored by Py_Initialize()
   https://bugs.python.org/issue34247

I fixed the first issue, I'm now working on the second one to see how
it can be fixed. Other option would be to backport the code from
master to the 3.7 branch, since the code in master has a way better
design. But it requires to backport a lot of changes. I'm not sure yet
what is the best option.

Victor


More information about the Python-Dev mailing list