On Mon, 23 Aug 2021 at 13:07, Guido van Rossum
But... I also care about backwards compatibility, and I have a crazy idea for making PyEval_GetLocals() work in a useful manner without compromising the behavior of the f_locals proxy:
- Let's start your idea of using the C-level f_locals field to store the "extra" variables. - The Python-level f_locals proxy looks in the actual frame "fast" locals and cells, and uses the C-level f_locals field only for extras - However, PyEval_GetLocals() doesn't return the proxy. - What PyEval_GetLocals() does: it calls PyEval_FastToLocals(), which makes a pass over the frame locals and adds them to the C-level f_locals field; then it returns that field. - So the borrowed reference is owned by the frame, which is the same as currently. - The proxy only uses the f_locals field for extra variables. If a variable is deleted in the frame but exists in the f_locals field, the proxy reports it as deleted. (This requires some care but can be done, since we have the mapping from proper variable names to frame locals or cells.) - We'll still deprecate PyEval_GetLocals() and PyEval_FastToLocals(), but unless the user turns the deprecation warning into an error, they will work for another few releases. Eventually we'll make PyEval_GetLocals() always return an error (similar to Mark's proposal), since it's in the stable ABI. - For PyEval_LocalsToFast() I don't care too much whether we keep it (per Mark's proposal) or make it return an error (per yours).
I don't think any of this is crazy, as it's how the PEP 558 reference implementation already works for individual keys (aside from only having a documented deprecation of PyEval_GetLocals(), not a programmatic one) You don't need to eliminate the cache (and hence break compatibility with PyEval_GetLocals()) to ensure it can never get out of sync - you just have to resync it every time you use it, rather than allowing the frame API consumer to make that call. The reason I don't like the proxy semantics proposed in PEP 667 is because it either makes the common case (analysing a frame from a tracing function while no application code is running) slower in order to avoid having to worry about cache consistency when trying to analyse a running frame, or else we have to write and maintain a whole lot more fast locals proxy specific code to implement the 5 different dict iteration APIs.
PS. The mapping from varname to position should be on the code object, not on the frame. This is how Mark does it (though his implementation would need to be extended to take cells into account).
It's taking cells into account that forces the lookup mapping to be on the frame: different executions of the same code object may reference different cell objects. Cheers, Nick. -- Nick Coghlan | ncoghlan@gmail.com | Brisbane, Australia