[New-bugs-announce] [issue47165] [C API] Test that the Python C API is compatible with C++

[STINNER Victor]

New submission from STINNER Victor <vstinner at python.org>:

There are more and more popular projects using the Python C API. The first big player is pybind11:
"Seamless operability between C++11 and Python"
https://pybind11.readthedocs.io/

Recently, I proposed a PR to add Python 3.11 support to the datatable project:
https://github.com/h2oai/datatable/pull/3231

My PR uses pythoncapi_compat.h header file which provides recent C API functions to old Python functions. The header file implements these functions as static inline function.

Problem: a static inline function implemented in a C header file used in a C++ code file can emit C++ compiler warnings.

In datatable, I got two kinds of C++ compiler warnings:

* Usage of the C NULL constant: C++ prefers nullptr
* "Old-style" cast like (PyObject*)obj: C++ prefers static_cast, reinterpret_cast, etc.

It seems like these compiler warnings are not enabled by default. The datatable project seems enabling them in its CI and I was asked to fix these warnings.

In the pythoncapi-compat project (*), I chose to use nullptr and reinterpret_cast if the "__cplusplus" macro is defined. Example:

---
// C++ compatibility
#ifdef __cplusplus
#  define PYCAPI_COMPAT_CAST(TYPE, EXPR) reinterpret_cast<TYPE>(EXPR)
#  define PYCAPI_COMPAT_NULL nullptr
#else
#  define PYCAPI_COMPAT_CAST(TYPE, EXPR) ((TYPE)(EXPR))
#  define PYCAPI_COMPAT_NULL NULL
#endif

// Cast argument to PyObject* type.
#ifndef _PyObject_CAST
#  define _PyObject_CAST(op) PYCAPI_COMPAT_CAST(PyObject*, op)
#endif
---

(*) https://github.com/python/pythoncapi_compat


It's unclear to me if the Python C API has or has not the same issue than pythoncapi_compat.h.


Last years, some old macros of the Python C API have been converted to static inline functions, like Py_INCREF(). It's unclear to me if these compiler warnings happen on Py_INCREF(). I don't understand why, but static inline macros from Python.h didn't emit compiler warnings in datatable, whereas similar static inline functions of pythoncapi_compat.h emitted compiler warnings.

Maybe there is a difference between <Python.h> and "Python.h". Or maybe it depends if the header file is a "local" file, or a "system" header file (ex: installed in /usr/include/ on Linux).


A first step would be to build a C++ extension as part of the Python test suite and check that there is no compiler warning. My GH-32175 PR is a proof-of-concept of that.

I don't know which C++ version we should target. pybind11 targets C++11. See bpo-39355 for a discussion about C++20: usage of the C++20 "module" keyword... which is a "contextual keyword" in practice.

----------
components: C API
messages: 416359
nosy: vstinner
priority: normal
severity: normal
status: open
title: [C API] Test that the Python C API is compatible with C++
type: enhancement
versions: Python 3.11

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue47165>
_______________________________________