[New-bugs-announce] [issue42990] Improve the C code for calling Python code

[Mark Shannon]

New submission from Mark Shannon <mark at hotpy.org>:

Currently, to make a call to Python (modules, classes, etc, not just functions) from C has to use the monster that is `_PyEval_EvalCode`.
As Python has adding features over the years, _PyEval_EvalCode has grown and grown. It is time for a refactor.

`_PyEval_EvalCode` takes 16, yes sixteen parameters!
Those 16 parameters fall into two main groups, one describing the function being called, and the other describing the arguments to the call.
Due to the jumbo size and complexity of _PyEval_EvalCode, we also have specialised forms of it, e.g. _PyFunction_Vectorcall that handle common cases and then bail to _PyEval_EvalCode in other cases.

In outline _PyEval_EvalCode performs the following actions:
1. Make a new frame.
2. Fill in that frame using the arguments and defaults provided.
3. If the code object has flags set for a generator, or coroutine, return a new generator or coroutine.
4. Otherwise call the interpreter with the newly created frame.

As _PyEval_EvalCode or its earlier equivalents have gained arguments, they have a left of list of legacy functions. It is not clear what is the "correct" function to use in C extensions. Hopefully extension code uses the `PyObject_Call` API, but `PyEval_EvalCodeEx` is part of the C-API.


To improve this situation, I propose:

A new C struct, the "frame descriptor" which contains the code object, defaults, globals, and names that describe the code to be executed. `PyFunctionObject` would wrap this, to simplify the common case of calling a Python function. 

Split the Eval functions into vector and tuple/dict forms, both forms taking a "frame descriptor", as well as the arguments.

Mark the older forms of the Eval functions as "legacy", creating a local "frame descriptor" and transforming the arguments in vector or tuple/dict forms, in the legacy functions.

Factor out the frame creation part of the Eval functions.


The above changes will be necessary for PEP 651, but I think would be a worthwhile improvement even if PEP 651 is not accepted.

----------
assignee: Mark.Shannon
components: Interpreter Core
messages: 385432
nosy: Mark.Shannon
priority: normal
severity: normal
stage: needs patch
status: open
title: Improve the C code for calling Python code

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue42990>
_______________________________________