ISTM this is better discussed on a case-by-case basis than as a blanket policy change. (The latter could end up causing a flood of trivial PRs from wannabe-contributors who found and fix yet another violation of the policy, which is both a nuisance for reviewers and a risk of introducing bugs due to being over-zealous.) On Mon, Feb 7, 2022 at 7:59 AM Victor Stinner <vstinner@python.org> wrote:
Hi,
I would like to replace runtime checks on C function arguments with assertions. For example, replace:
PyObject * PyDict_GetItemWithError(PyObject *op, PyObject *key) { if (!PyDict_Check(op)) { PyErr_BadInternalCall(); return NULL; } ... }
with:
PyObject * PyDict_GetItemWithError(PyObject *op, PyObject *key) { assert(PyDict_Check(op)); ... }
I'm not asking to remove all of them. For example, PyList_GetItem(list, index) will continuing raising an IndexError if the index is out of range.
I'm asking to replace runtime checks with assertions when the C API is "obviously" misused: replace PyErr_BadInternalCall(), _Py_CheckFunctionResult() and _Py_CheckSlotResult() with assertions. The exact scope should be defined.
--
Python tries to be nice to C extensions authors and calls PyErr_BadInternalCall() to emit a nice SystemError exception when the C API is misused. There are many sanity checks run at runtime. We pay the cost of these runtime checks even if a C extension is well written (has no bug).
In Python 3.6, I added the _Py_CheckFunctionResult() function to check the result of any function call. A function must return a non-NULL object and not raises an exception, or return NULL and raises an exception. Otherwise, SystemError is raised. For example, PyObject_Call() uses it to check the PyTypeObject.tp_call result.
In Python 3.10, I added a similar _Py_CheckSlotResult() function to check the result of slot functions. For example, PyObject_Size() checks the result of the PyTypeObject.tp_as_sequence.sq_length slot function.
--
I modified Python 3.8 to be able to use a debug Python build as a drop-in replacement of a Python release build: debug and release builds are now ABI compatible. It is no longer needed to rebuild C extensions in debug mode. My first goal was to be able to use gdb on a Python binary built with -O0:
https://developers.redhat.com/articles/2021/09/08/debugging-python-c-extensi...
A debug Python build adds many other debug features useful to develop C extensions: https://docs.python.org/dev/using/configure.html#python-debug-build
IMO with Python 3.11, it's now easy enough to get a Python debug build to develop or debug your C extensions. End users should not have to pay the price of these runtime checks.
On most Linux distribution, a Python debug build is available as "python3-debug" executable. To check if you are using a debug build, check if the sys.gettotalrefcount() function is available.
By the way, it is also possible to build Python in release mode with assertions using "./configure --with-assertions": https://docs.python.org/dev/using/configure.html#cmdoption-with-assertions
So, what do you think?
--
In 2019, I already proposed this idea on the bug tracker, but I abandoned my idea: https://bugs.python.org/issue37406
Victor -- Night gathers, and now my watch begins. It shall not end until my death. _______________________________________________ Python-Dev mailing list -- python-dev@python.org To unsubscribe send an email to python-dev-leave@python.org https://mail.python.org/mailman3/lists/python-dev.python.org/ Message archived at https://mail.python.org/archives/list/python-dev@python.org/message/RR3DJ4IZ... Code of Conduct: http://python.org/psf/codeofconduct/
-- --Guido van Rossum (python.org/~guido) *Pronouns: he/him **(why is my pronoun here?)* <http://feministing.com/2015/02/03/how-using-they-as-a-singular-pronoun-can-c...>