Petr Viktorin wrote:
Q)
- What do you mean by "core binary"?
- If Python is embedded into another application, what happens to the state?
- If the "main" interpreter is stopped and re-initialized (Py_FinalizeEx() Py_InitializeEx()), is the state recreated?
A)
- The term "core binary" refers to the binary file containing the compiled result of "pythoncore".
- Nothing changes other than which binary is used to allocate the object (C global variable)
- Generally speaking, Py_InitializeEx()/Py_FinalizeEx() denounces the lifetime of the core binary.
I'm afraid I'll need to ask you to be more exact. A "binary file" doesn't really have a lifetime (unless you count "from creation until deleting from the disk", which isn't helpful). Can I assume you mean the process? You can call Py_InitializeEx()/Py_FinalizeEx() multiple times in a single process. So to be clear, which one should be the lifetime of "Core state"? A. from Py_InitializeEx() to Py_FinalizeEx() B. from first Python initialization to the process shutdown
- The lifetime I was referring to for a binary file is from when it's loaded into memory, to the moment it's unloaded.
- Since you stated that "Py_InitializeEx"/"Py_FinalizeEx()" can be invoked multiple times, Core State's lifetime is tied to those calls. (Option A)
- This also means Core State can not be a global static storage item, but a heap allocated object.
Q) So, if some Python code modifies sys.executable, the change should affect all interpreters in the process? A) No, Core State can not hold any Python heap allocated objects, also, virtually all Python first party types COPIES data into newly allocated memory. Q)
- What does "Branches from Core State" mean?
- How will this state be available to the interpreter it's tied to?
A) pointer to the active Interpreter State.
- This casually means that access to the Core State can be done with just a
Note: This can be accomplished via either a constant pointer inside the InterpreterState or (Global State only) with a global C variable.
- See the above answer but from the Thread State & Interpreter State branching viewpoint. Note: The active Thread State assumes we're not changing how threading works in Python, BUT it's flexible enough for thread_local storage.
Q) How is this different from C "static" storage? A) Generally speaking, it's not different at all. This is the only State definition that can be dropped all together if widely unpopular. Note: But on the flip side, we will still require an initialization/finalization for extensions to implement (see Windows WSA/MySQL documentation) Do you have a link to that documentation? I'm not familiar enough with these to find the relevant docs myself. (To be honest I'm not sure what WSA is.) Anyway, I would like to avoid extra initialization/finalization hooks (which seem necessary for "Extension state"), and base as much as possible on the regular PyObject* lifecycle (of the module objects). Running Python code in interpreter shutdown is hard to get right, especially if you allow references to Python objects and so you need the full GC machinery.
After reviewing things over, I realized Winsock and MySQL don't fit the bill as I thought they did, BUT, that doesn't say that we don't need extension specific initialization/finalization, I just haven't found a library that requires it.