
On 23. 02. 22 2:46, Eric Snow wrote:
Thanks for the responses. I've replied inline below.
Same here :)
Immortal Global Objects -----------------------
All objects that we expect to be shared globally (between interpreters) will be made immortal. That includes the following:
* singletons (``None``, ``True``, ``False``, ``Ellipsis``, ``NotImplemented``) * all static types (e.g. ``PyLong_Type``, ``PyExc_Exception``) * all static objects in ``_PyRuntimeState.global_objects`` (e.g. identifiers, small ints)
All such objects will be immutable. In the case of the static types, they will be effectively immutable. ``PyTypeObject`` has some mutable start (``tp_dict`` and ``tp_subclasses``), but we can work around this by storing that state on ``PyInterpreterState`` instead of on the respective static type object. Then the ``__dict__``, etc. getter will do a lookup on the current interpreter, if appropriate, instead of using ``tp_dict``.
But tp_dict is also public C-API. How will that be handled? Perhaps naively, I thought static types' dicts could be treated as (deeply) immutable, and shared?
They are immutable from Python code but not from C (due to tp_dict). Basically, we will document that tp_dict should not be used directly (in the public API) and refer users to a public getter function. I'll note this in the PEP.
What worries me is that existing users of the API haven't read the new documentation. What will happen if users do use it? Or worse, add things to it? (Hm, the current docs are already rather confusing -- 3.2 added a note that "It is not safe to ... modify tp_dict with the dictionary C-API.", but above that it says "extra attributes for the type may be added to this dictionary [in some cases]") [...]
And from the other thread:
On 17. 02. 22 18:23, Eric Snow wrote:
On Thu, Feb 17, 2022 at 3:42 AM Petr Viktorin <encukou@gmail.com> wrote:
Weren't you planning a PEP on subinterpreter GIL as well? Do you want to submit them together?
I'd have to think about that. The other PEP I'm writing for per-interpreter GIL doesn't require immortal objects. They just simplify a number of things. That's my motivation for writing this PEP, in fact. :)
Please think about it. If you removed the benefits for per-interpreter GIL, the motivation section would be reduced to is memory savings for fork/CoW. (And lots of performance improvements that are great in theory but sum up to a 4% loss.)
Sounds good. Would this involve more than a note at the top of the PEP?
No, a note would work great. If you read the motivation carefully, it's (IMO) clear that it's rather weak without the other PEP. But that realization shouldn't come as a surprise to the reader.
Having thought about it some more, I don't think this PEP should be strictly bound to per-interpreter GIL. That is certainly my personal motivation. However, we have a small set of users that would benefit significantly, the change is relatively small and simple, and the risk of breaking users is also small. In fact, we regularly have more disruptive changes to internals that do not require a PEP.
Right, with the recent performance improvements it's looking like it might stand on its own after all.
So it seems like the bar should be pretty low for this one (assuming we get the performance penalty low enough). If it were some massive or broadly impactful (or even clearly public) change then I suppose you could call the motivation weak. However, this isn't that sort of PEP. Honestly, it might not have needed a PEP in the first place if I had been a bit more clear about the idea earlier.
Maybe it's good to have a PEP to clear that up :)