On Thu, Aug 2, 2018 at 3:50 AM Victor Stinner
I'm still doing further bug fixes and cleanup in the master branch: https://bugs.python.org/issue34170
I'm doing more and more changes.
Yeah, it's a question of what you plan to backport. As Barry suggested, it would be great if you had a WIP PR for the backport, just so Ned (and others) has a point of reference.
I just added two new files: Include/coreconfig.h and Python/coreconfig.h. IMHO it's better to put similar code in separated files.
FYI Python/coreconfig.c has 954 files and Include/coreconfig.h has 319 lines.
Nick might have a better opinion, particularly when in comes to a C codebase, but I'm in favor of keeping separate things separate, and especially when in relates to the runtime. (Historically we haven't been great about considering the runtime as a distinct part of CPython.) Hence +1 to keeping the runtime config separate, especially given the size of the files. :) Presumably pystate.c (and pystate.h) got smaller by roughly the same amount? Also, would it make sense for at least some of coreconfig.h to live in Include/internal, either as coreconfig.h or in the internal pystate.h?
I'm ok to rename the structure and the files if you prefer a different name.
I'd love to see all the "core" names changed to "runtime", in the same way that PEP 432 was updated. It was a point of confusion in the PEP until we changed the name, and doing so helped. I thought we had also changed the code, but apparently not. For that matter, I'd love to see PEP 432 and the codebase synced up. While the overall plan is still consistent, a number of details (e.g. the intent and content of the "core" config) have diverged.
About that, I'm working on a subproject: abandon Py_xxx global configuration variables to replace them with accessing interpreter->core_config->xxx.
+1 IMHO it's a natural consequence of having a runtime/core config. In fact, having both is problematic because it's easy for us to accidentally ignore a global-var-as-config (prior to Py_Initialize); the relationship between those global variables and runtime initialization isn't very obvious in the code (unlike the config struct). It's also confusing for embedders if we have both. As I've already expressed, I'm definitely in favor of improving encapsulation of the runtime (and moving away from globals). :) Note that there are backward compatibility issues to deal with. AFAIU if we start ignoring those global variables during initialization then it's going to cause problems for embedders. If we get rid of the variables altogether then it would break extension modules that currently rely on reading those parts of the runtime config. So I'm guessing you planned on deprecating any use of those global variables and, in the spirit of your goals for the C-API, provide a public API for extensions to access the info in the runtime config instead. FWIW, I recall that Nick and I discussed this relative to PEP 432 a while ago and remember the decision to stay with the status quo for now (to avoid scope creep in the PEP). Apparently that consideration did not get recorded in the PEP (that I could see with a quick skim of the text). The mailing lists may have the discussion somewhere.
I'm not sure yet if it's a good idea or not, but it would allow to have two interpreters to have their own different configuration. Imagine two interpreters with different sys.path running in isolated mode. Or maybe an interpreter without importlib?
Yeah, a number of interesting possibilities open up as we further encapsulate the runtime and move away from globals.
One of the issue is that we have now two copies of the same option. For example, Py_BytesWarningFlag and interpreter->core_config->bytes_warning. That's why I would like to switch to core_config.
+1
But I'm also trying to make sure that the two variables have the same value: https://github.com/python/cpython/commit/a4d20b2e5ece2120f129cb4dda951a6c246...
Yep. That is necessary while the global config variables still exist. It's just risky since it's easy for us to change the config but forget to update the global config vars (that are shadowing the config). It would probably be worth making sure we have tests that verify that the two stay synchronized. -eric