On Fri, 30 Oct 2020 at 23:34, Victor Stinner <vstinner@python.org> wrote:
Le ven. 30 oct. 2020 à 11:02, Nick Coghlan <ncoghlan@gmail.com> a écrit :
Ok, I've created https://bugs.python.org/issue42197 to track it.
Please also have a look at PEP 558 and its draft reference implementation at https://github.com/python/cpython/pull/3640
I don't think that the PEP 558 and bpo-42197 are incompatible.
Debuggers and profilers usually only care of specific frames or function calls (ex: 10% of function calls or even a single function call in a whole application). The problem is how to make them as efficient as possible for "no operation" calls, when they don't care about the current frame. Avoiding PyFrame_FastToLocalsWithError() to enter the debugger/profile and avoiding PyFrame_LocalsToFast() on exit sounds a simple and practical solution.
Aye, I agree. I just don't think we can remove those implicit calls without preparing a replacement API first.
By the way, I created https://bugs.python.org/issue40421 to prepare the C API to make the PyFrameObject structure opaque. Once it will be opaque, we will have more freedom on the API exposed to inspect frame locals.
IMO it's a good idea to require function calls to inspect frame locals, and not let developers think that PyFrameObject.f_locals is always up-to-date and can always be modified.
Aye, if direct access to the PyFrameObject.f_locals struct field goes away, then the cache refresh can be added to the accessor APIs, just as PEP 558 proposes doing for Python code.
The good news is that PyFrame_GetLocals() and PyFrame_SetLocals() can easily be reimplemented in Python 3.9 for my compatibility header file: https://github.com/pythoncapi/pythoncapi_compat
Such API avoids a complex proxy and simply reuses a regular dict object (exiting PyFrameObject.f_locals).
Unfortunately, cell variables mean that there's no way to make snapshot-with-writeback logic consistently correct in the presence of generators and coroutines (see https://www.python.org/dev/peps/pep-0558/#resolving-the-issues-with-tracing-... for the gist of that problem). The early iterations of PEP 558 worked that way (like you, I wanted to avoid having to actually implement the write-through proxy), but Nathaniel kept finding holes where bugs like bpo-30744 could still happen (hence the acknowledgement in the PEP), so I eventually conceded defeat and accepted that the proxy was the only approach that could ever be truly correct. The core of the write-through proxy implementation isn't actually too bad: https://github.com/python/cpython/blob/ed6e53be7794ec94924b69cd46d5b009633c6... The annoying parts are the outright copy-and-paste of odict code at the end of the file, and the few remaining mutable mapping methods that still raise NotImplementedError rather than doing what you'd expect. Cheers, Nick. -- Nick Coghlan | ncoghlan@gmail.com | Brisbane, Australia