On Tue, Feb 18, 2020 at 11:09 AM Serhiy Storchaka <storchaka@gmail.com> wrote:
18.02.20 20:13, Guido van Rossum пише:
> I am a little confused how you get from "there are extra frames in the
> traceback" to "modify exec() to run code in another frame".

Sorry for skipping several steps. I'll try to expand them here. PEP 558
does not have relation to this, as frame is not the part of locals().

Ah, fair.

exec() is used by the import machinery to execute the module code.
Currently frames look like this:

     import foo  # user code
     # many frames in the import machinery
     # ...
     # ...
     exec(code, module.__dict__)  # still in the import machinery
     # user code in foo.py

I propose to add a parameter to exec() which will allow to execute the
code in the imported mode as exec() was called immediately from the
importer code:

     import foo  # user code
     # user code in foo.py

As it was when the import machinery was implemented in C.

This makes sense (except when debugging the import machinery, which is also a valid use case. :-).

The caller would still have to provide a dict for the globals (the module dict), and I guess it would have to point to the *previous* frame (the one executing `import foo`) so that exec() can create a new frame pointing back there. And somehow we would need to set things up so that after exec() completes or raises we automatically return or raise to the parent frame, rather than passing control to exec()'s caller. Or alternatively, we'd need exec()'s caller (deep down inside importlib) to be able to say "pretend to return or raise from this (new) frame".
 
I did not propose to modify exec() to run code in another frame, I
proposde to modify exec() to run code in new frame whose parent is a
specified frame (either arbitrary frame or on of ancestors of the caller
frame).

I understand that now.

I think it may be not difficult to do. I ask whether there are other
applications of this or similar feature and what form of the feature
will be more convenient.

My gut feeling says that the only other use cases will be alternatives to importlib, perhaps other kinds of import hooks.
 
It just occurred to me that it may be not a feature of exec(), but a
separate function which will allow to execute an arbitrary function, not
just a Python code, as it was called from the different frame.

     apply_in_frame(frame, func, /, *args, **kwargs)

or

     apply_with_frames_skipped(stacklevel, func, /, *args, **kwargs)

The question is how to call this function and where to put it. In the
first variant it could be a method of frame.

So either way it would create a new frame, right? And it would not pass control to the caller, but to the indicated frame.

I'm not that keen on passing a stacklevel -- that could easily cause confusion, and to calculate the right level the caller would have to crawl up the stack anyway.

A problem I see with either solution is that it looks like except and finally blocks in the intervening frames would be skipped. Or perhaps not, but at that point I suspect that the implementation won't be all that easy after all.

So *if* we end up doing this optimization I think it may be best to make it something that's clearly meant for the importlib use case. Sine we already have sys._getframe(), maybe it could be another magic thing in sys?

--
--Guido van Rossum (python.org/~guido)