capi-sig June 2020

capi-sig@python.org

15 participants
12 discussions

Metaclasses and Limited API
by Sebastian Berg 10 Jun '20

10 Jun '20

Hi, Since there is a discussion about the limited API and headers. A long time ago I asked about this on python-dev: It concerns me a bit that the limited API does not seem to support MetaClasses in PyType_FromSpec (or FromSpecWithBase). In particular, last time I checked, it seemed impossible to extend the type struct within such a metaclass [1]. There also seemed to be an issue with PyType_FromSpecWithBases allocating exactly a base type-object, instead of using `tp_alloc` as `PyType_Type.tp_new` would. This and how to efficiently cache Python objects in methods are the two main concerns I have right now for long-term full adoption of the limited API. (There are other difficulties, but for the ones I am aware, I at least have an idea how to they need to be addressed). I suppose it may actually be possible to hack around the FromSpec issues, although I am not sure right now. unless subclassing PyType_Type in C is for some reason different and very much discouraged? Cheers, Sebastian [1] And e.g. PySide seemed to "do" this, and got away with it, because there is an additional NULL at the end, as a termination for slots or so. Which for them is unused and they just happen to use.

3 5

How can we ship a header file, or even a dynamic library, to port existing C extension modules to the new incompatible Python C API?
by Victor Stinner 09 Jun '20

09 Jun '20

Hi, As I wrote in a previous email, the C API of CPython is evolving slowly to hide more and more implementation details. Sadly, some changes break the backward compatibility. Sometimes, a new function must be used, except that the function doesn't exist in older Python versions. "Update on C API changes to hide implementation details" https://mail.python.org/archives/list/capi-sig@python.org/thread/HYFQHLGNCR… Would it be possible to ship a header file (a single ".h" file) and/or even a dynamic library to provide new functions to old Python versions? I'm not sure if it's possible to ship a header file as a package on PyPI and make it available for C compilers easily without monkey patching distutils. Another option is to encourage Linux distributions to package such header file to make it easy to install on the system. Another (worse?) option is to advise authors of each C extension module to vendor a copy of the header file in their project and update it from time to time when needed. At least in C, we can provide most "missing" functions on older Python versions as macros or static inline functions. Python 3.6 now requires a subset C99 with static inline functions. The situation is very similar to the Python 2 to Python 3 transition. Benjamin Peterson wrote the "six" which became quickly very popular. First, projects vendored a copy of the "six" module as a submodule of their project (src/project/six.py, imported as "project.six"). Then, slowly, the "six" module was shipped by Linux distributions or installed via "pip install six". Cython already does that internally. A few examples: https://github.com/cython/cython/blob/57b4bdeb88cc18f8e2f89fe9a7cf506d4ee0f… #if PY_VERSION_HEX < 0x030200A4 typedef long Py_hash_t; #define __Pyx_PyInt_FromHash_t PyInt_FromLong #define __Pyx_PyInt_AsHash_t __Pyx_PyIndex_AsHash_t #else #define __Pyx_PyInt_FromHash_t PyInt_FromSsize_t #define __Pyx_PyInt_AsHash_t __Pyx_PyIndex_AsSsize_t #endif #if CYTHON_COMPILING_IN_LIMITED_API #define __Pyx_PyThreadState_Current PyThreadState_Get() #elif !CYTHON_FAST_THREAD_STATE #define __Pyx_PyThreadState_Current PyThreadState_GET() #elif PY_VERSION_HEX >= 0x03060000 //#elif PY_VERSION_HEX >= 0x03050200 // Actually added in 3.5.2, but compiling against that does not guarantee that we get imported there. #define __Pyx_PyThreadState_Current _PyThreadState_UncheckedGet() #elif PY_VERSION_HEX >= 0x03000000 #define __Pyx_PyThreadState_Current PyThreadState_GET() #else #define __Pyx_PyThreadState_Current _PyThreadState_Current #endif #if PY_VERSION_HEX >= 0x030900A4 #define __Pyx_SET_REFCNT(obj, refcnt) Py_SET_REFCNT(obj, refcnt) #define __Pyx_SET_SIZE(obj, size) Py_SET_SIZE(obj, size) #else #define __Pyx_SET_REFCNT(obj, refcnt) Py_REFCNT(obj) = (refcnt) #define __Pyx_SET_SIZE(obj, size) Py_SIZE(obj) = (size) #endif Victor -- Night gathers, and now my watch begins. It shall not end until my death.

6 22

2022

2021

2020

2019

2018

2017

2016

2015

2014

2013

2012

2011

2010

2009

2008

2007

capi-sig June 2020