PEP 558, the simplest thing I could come up with

Hi Nick, Our discussion on PEP 558 got me thinking "What is the simplest thing that would work?". This is what I came up (in the form of a draft PEP): https://github.com/markshannon/peps/blob/pep-locals/pep-06xx.rst It doesn't have O(1) len(f_locals), and it does break `PyEval_GetLocals()` but I think the that is a small price to pay for simplicity and consistency. What do you think? Cheers, Mark.

On Fri, 30 Jul 2021, 6:05 am Mark Shannon, <mark@hotpy.org> wrote:
I don't think it is OK to break PyEval_GetLocals() when we really don't need to, and the proposal also discards all the feedback that I received on earlier iterations of PEP 558. (I particularly recommend reading Nathaniel's analysis of why returning the proxy from locals() would be more likely to cause bugs in existing code than it would be to eliminate any). As it looks like we have some fairly fundamental disagreements regarding the level of behaviour change and API breakage that is acceptable for the sake of a simpler implementation, I think we're going to need competing PEPs for the SC to evaluate (or delegate to Nathaniel for evaluation) if you want to pursue that path further. (I'm only OK with explicitly breaking PyEval_LocalsToFast because the write-back strategy is already intrinsically broken. By contrast, maintaining a frame cache for optimised frames isn't fundamentally flawed, it's just a matter of figuring out when and how to pay the cost of updating it to avoid wasting those CPU cycles in code that doesn't care if the cache is up to date or not) Cheers, Nick.

On Thu, Jul 29, 2021 at 4:52 PM Nick Coghlan <ncoghlan@gmail.com> wrote:
Heh, I was actually just re-reading PEP 558 and going to ask you to include more details to justify the complexity, as compared to something like Mark's latest proposal here -- I'd totally forgotten I wrote that old post :-). So that was a timely reminder! Looking at the references in the PEP, is this the writeup you're talking about? https://mail.python.org/pipermail/python-dev/2019-May/157738.html The conclusion there is:
I *think* (please correct me if I'm wrong) that what that calls [PEP-minus-tracing] is now corresponds to the current PEP draft, and [proxy] corresponds to Mark's draft at the beginning of this thread? -n -- Nathaniel J. Smith -- https://vorpus.org

Bringing the public record up to date with a brief off-list discussion between Mark, Nathaniel and I: * Mark hasn't convinced me that getting rid of the frame value cache entirely for optimised frames is a good idea, so he's going to write that proposal up as a competing PEP. Once it has been drafted and is ready for review, he will request the SC assign a PEP delegate. * On the PEP 558 front, Mark's proposal has highlighted a few inefficiencies in my reference implementation, where the code still implicitly updates the frame value cache in cases where the cache being up to date may not matter to the proxy API client. So I'll be working on another iteration of the implementation that ensures each caching proxy instance (at worst) only pays the O(N) cache refresh price on the first less than O(N) operation that relies on the cache being up to date, rather than paying it every time "f_locals" is retrieved from the frame object. We still have plenty of time before 3.11b1, so we expect it will be a month or two before the two proposals are in a position to be compared directly. Cheers, Nick. On Fri, 30 Jul 2021, 5:25 pm Nick Coghlan, <ncoghlan@gmail.com> wrote:

On Fri, 30 Jul 2021, 6:05 am Mark Shannon, <mark@hotpy.org> wrote:
I don't think it is OK to break PyEval_GetLocals() when we really don't need to, and the proposal also discards all the feedback that I received on earlier iterations of PEP 558. (I particularly recommend reading Nathaniel's analysis of why returning the proxy from locals() would be more likely to cause bugs in existing code than it would be to eliminate any). As it looks like we have some fairly fundamental disagreements regarding the level of behaviour change and API breakage that is acceptable for the sake of a simpler implementation, I think we're going to need competing PEPs for the SC to evaluate (or delegate to Nathaniel for evaluation) if you want to pursue that path further. (I'm only OK with explicitly breaking PyEval_LocalsToFast because the write-back strategy is already intrinsically broken. By contrast, maintaining a frame cache for optimised frames isn't fundamentally flawed, it's just a matter of figuring out when and how to pay the cost of updating it to avoid wasting those CPU cycles in code that doesn't care if the cache is up to date or not) Cheers, Nick.

On Thu, Jul 29, 2021 at 4:52 PM Nick Coghlan <ncoghlan@gmail.com> wrote:
Heh, I was actually just re-reading PEP 558 and going to ask you to include more details to justify the complexity, as compared to something like Mark's latest proposal here -- I'd totally forgotten I wrote that old post :-). So that was a timely reminder! Looking at the references in the PEP, is this the writeup you're talking about? https://mail.python.org/pipermail/python-dev/2019-May/157738.html The conclusion there is:
I *think* (please correct me if I'm wrong) that what that calls [PEP-minus-tracing] is now corresponds to the current PEP draft, and [proxy] corresponds to Mark's draft at the beginning of this thread? -n -- Nathaniel J. Smith -- https://vorpus.org

Bringing the public record up to date with a brief off-list discussion between Mark, Nathaniel and I: * Mark hasn't convinced me that getting rid of the frame value cache entirely for optimised frames is a good idea, so he's going to write that proposal up as a competing PEP. Once it has been drafted and is ready for review, he will request the SC assign a PEP delegate. * On the PEP 558 front, Mark's proposal has highlighted a few inefficiencies in my reference implementation, where the code still implicitly updates the frame value cache in cases where the cache being up to date may not matter to the proxy API client. So I'll be working on another iteration of the implementation that ensures each caching proxy instance (at worst) only pays the O(N) cache refresh price on the first less than O(N) operation that relies on the cache being up to date, rather than paying it every time "f_locals" is retrieved from the frame object. We still have plenty of time before 3.11b1, so we expect it will be a month or two before the two proposals are in a position to be compared directly. Cheers, Nick. On Fri, 30 Jul 2021, 5:25 pm Nick Coghlan, <ncoghlan@gmail.com> wrote:
participants (4)
-
Brett Cannon
-
Mark Shannon
-
Nathaniel Smith
-
Nick Coghlan