[Python-Dev] New _Py_InitializeFromConfig() function (PEP 432)

[Victor Stinner]

It seems like the PEP 432 proposes an API designed from scratch as the
target API. I started from the 28 years old C code and I tried to
cleanup the code. Our method is different, so it's not surprising that
the result is different :-) My intent is to get:

* a function to read *all* configuration with no side effect and put
it into a single structure: _PyCoreConfig
* modify Py_Main() and all variants of Py_Initialize() to always end
in the same code path using _PyCoreConfig

I'm open to change to move the current implementation closer to the
PEP 432. But it seems like I don't understand well the subtle parts of
this PEP.


> * core/runtime config
>   + minimal

I'm not sure of what you mean by "minimum". I collected *all*
parameters need to initialize Python and there are something like 40
parameters or more. But _PyCoreConfig has enough parameters to
initialize a full Python with a working importlib and a REPL, for
example.

Where do you put the limit for "minimal"?


> * main interpreter config
>   + includes everything needed to finish full runtime initialization

For practical reason, I prefer to be able to pass the "path
configuration" at the C level to be able to initialize importlib.

IMHO it makes the current code base simpler, since the path
computation is fully implemented in C.
For example, it allows embedders to use a fixed sys.path in C. IMHO a
good example is to imagine a Python runtime with *no* filesystem
access, where everything is built into the binary. So we have to skip
completly the code computing the path configuration, since this
operating access the filesystem as well!

Maybe something should be changed here?


> The duplication is due to there being C and PyObject versions.  It is
> for the sake of embedders (and a little bit of sanity).  The big
> reason why it shouldn't be a problem is because
> PyMainInterpreterConfig is generated directly from PyRuntimeConfig
> (AKA PyCoreConfig) and only *after* we've used the runtime config to
> bootstrap the limited runtime (after which it shouldn't be modified
> ever).  So there's no risk of inconsistency, right?

Currently, core_config and main_config can be modified, as global
variables: some parameters exist in 3 versions, each is modified. And
it's unclear which one has the highest priority. For example, if we
decide to always rely on core_config, we have to modify the C code to
not longer access Py_VerboseFlag after Py_Initialize().

I'm talking about the current C code, not your theorical API.


> Perhaps it would make sense to only keep a const copy of both, to
> avoid modification?

Maybe. But currently, some flags are modified after Py_Initialize(),
especially Py_InspectFlag in main.c.

I would prefer to keep a read-only configuration to reflect what
sys.flags contains and know how Python has been initialized.


> As noted above, the core/runtime config should probably be on
> PyRuntimeState instead.

For me it doesn't make sense to put all _PyCoreConfig parameters into
PyRuntimeState.

PyRuntimeState seems to be a singleton, so it means that all
interepreters would have the same configuration. Whereas I like the
idea of having a different verbose and/or sys.path per intepreter. Or
maybe I misunderstood what you mean by "core config". I'm talking
about the _current_ _PyCoreConfig in master.

Victor