[New-bugs-announce] [issue47241] [C API] Move the PyCodeObject structure to the internal C API (make the structure opaque)

[STINNER Victor]

New submission from STINNER Victor <vstinner at python.org>:

The PyCodeObject structure is documented at:
https://docs.python.org/dev/c-api/code.html

The structured evolved a lot in Python 3.11 to optimize Python/ceval.c performance:

* read-only co_code (object) was replaced with modifiable co_code_adaptive (char[])
* co_varnames, co_freevars and co_cellvars were removed: merged into co_localsplusnames with co_localspluskinds
* co_cell2arg was removed
* co_zombieframe ("free list") was removed
* co_opcache, co_opcache_map, co_opcache_flag and co_opcache_size were removed: see PEP 659
  https://peps.python.org/pep-0659/

New members:

* co_exceptiontable
* co_warmup, co_code_adaptive
* co_nlocalsplus, co_nplaincellvars, co_ncellvars, co_nfreevars
* co_localsplusnames, co_localspluskinds
* co_qualname
* co_endlinetable, co_columntable

The PyCodeObject structure should be made opaque in the public C API and only accessed with function calls.

In Python 3.11, the PyFrameObject structure was made opaque (bpo-46836) and multiple getters were added (bpo-40421). The idea is similar, but it might be too late to do that in Python 3.11 (beta1 feature freeze is close).

---

By the way, it was proposed multiple times on python-dev to mark the PyCode_New() function as "unstable" since its API changed often. PEP 670 "Python Positional-Only Parameters" caused a lot of troubles in Cython when it added a new parameter to PyCode_New(). The change was reverted: instead, a new PyCode_NewWithPosOnlyArgs() function was added.

* PyCode_New() has 19 parameters!
* PyCode_NewWithPosOnlyArgs() has 20 parameters!!

On Python 3.11a1 and newer, Cython uses the code.replace() method (added to Python 3.8) to build new code objects. Otherwise, it just calls directly PyCode_New().
https://docs.python.org/dev/library/types.html#types.CodeType.replace

----------
components: C API
messages: 416869
nosy: vstinner
priority: normal
severity: normal
status: open
title: [C API] Move the PyCodeObject structure to the internal C API (make the structure opaque)
versions: Python 3.11

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue47241>
_______________________________________