
Again, thanks for the reply. It's helpful. My further responses are inline below. -eric On Thu, Feb 17, 2022 at 3:42 AM Petr Viktorin <encukou@gmail.com> wrote:
Agreed. However, what behavior do users expect and what guarantees do we make? Do we indicate how to interpret the refcount value they receive? What are the use cases under which a user would set an object's refcount to a specific value? Are users setting the refcount of objects they did not create?
That's what I hoped the PEP would tell me. Instead of simply claiming that there won't be issues, it should explain why we won't have any issues. [snip] IMO, the reasoning should start from the assumption that things will break, and explain why they won't (or why the breakage is acceptable). If the PEP simply tells me upfront that things will be OK, I have a hard time trusting it.
IOW, it's clear you've thought about this a lot (especially after reading your replies here), but it's not clear from the PEP. That might be editorial nitpicking, if it wasn't for the fact that I want find any gaps in your research and reasoning, and invite everyone else to look for them as well.
Good point.. It's easy to dump a bunch of unnecessary info into a PEP, and it was hard for me to know where the line was in this case. There hadn't been much discussion previously about the possible ways this change might break users. So thanks for bringing this up. I'll be sure to put a more detailed explanation in the PEP, with a bit more evidence too.
Ah, I see. I was confused by this:
No worries! I'm glad we cleared it up. I'll make sure the PEP is more understandable about this.
This is also true even with the GIL, though the impact is smaller.
Smaller than what? The baseline for that comparison is a hypothetical GIL-less interpreter, which is only introduced in the next section. Perhaps say something like "Python's GIL helps avoid this effect, but doesn't eliminate it."
Good point. I'll clarify the point.
Weren't you planning a PEP on subinterpreter GIL as well? Do you want to submit them together?
I'd have to think about that. The other PEP I'm writing for per-interpreter GIL doesn't require immortal objects. They just simplify a number of things. That's my motivation for writing this PEP, in fact. :)
Please think about it. If you removed the benefits for per-interpreter GIL, the motivation section would be reduced to is memory savings for fork/CoW. (And lots of performance improvements that are great in theory but sum up to a 4% loss.)
Sounds good. Would this involve more than a note at the top of the PEP? And just to be clear, I don't think the fate of a per-interpreter GIL PEP should not depend on this one.
It wouldn't match _Py_IMMORTAL_REFCNT, but the high bit of _Py_IMMORTAL_REFCNT would still match. That bit is what we would actually be checking, rather than the full value.
It makes sense once you know _Py_IMMORTAL_REFCNT has two bits set. Maybe it'd be good to note that detail -- it's an internal detail, but crucial for making things safe.
Will do.
What about extensions compiled with Python 3.11 (with this PEP) that use an older version of the stable ABI, and thus should be compatible with 3.2+? Will they use the old versions of the macros? How will that be tested?
It wouldn't matter unless an object's refcount reached _Py_IMMORTAL_REFCNT, at which point incref/decref would start noop'ing. What is the likelihood (in real code) that an object's refcount would grow that far? Even then, would such an object ever be expected to go back to 0 (and be dealloc'ed)? Otherwise the point is moot.
That's exactly the questions I'd hope the PEP to answer. I could estimate that likelihood myself, but I'd really rather just check your work ;)
(Hm, maybe I couldn't even estimate this myself. The PEP doesn't say what the value of _Py_IMMORTAL_REFCNT is, and in the ref implementation a comment says "This can be safely changed to a smaller value".)
Got it. I'll be sure that the PEP is more clear about that. Thanks for letting me know.