Mailman 3 June 2020 - capi-sig

ANN: expy 0.6.5 released!
by Yingjie Lan April 16, 2022

April 16, 2022

EXPY is an express way to extend Python! EXPY provides a way to extend python in an elegant way. For more information and a tutorial, see: http://expy.sourceforge.net/ What's new: 1. Correct treatment of __init__ method. 2. Give warnings of missing Py_INCREF on appropriate special type methods. 3. Documentation update. Cheers, Yingjie

3 3

ANNOUNCE: PyBindGen 0.8
by Gustavo Carneiro March 10, 2022

March 10, 2022

The other day I released PyBindGen 0.8. Main news is that it features a new experimental header file scanner based on pygccxml (i.e., it's similar to py++ in scope, if not in maturity, but does not use boost.pythonunderneath). == What == PyBindGen is a Python module that is geared to generating C/C++ code that binds a C/C++ library for Python. It does so without extensive use of either C++ templates or C pre-processor macros. It has modular handling of C/C++ types, and can be easily extended with Python plugins. The generated code is almost as clean as what a human programmer would write, and does not depend on any library or header files besides Python itself. == Where == https://launchpad.net/pybindgen/ == NEWS == - Support C++ instance attributes through getter/setter methods - Support functions as methods of C++ classes - Support the PyObject* type - Support unsigned int, C strings (char*) (from Mark Lee) - Add basic support for enum types - New experimental automatic module generator based on C/C++ header file scanner and annotations in comments, using pygccxml - Some bug fixes -- Gustavo J. A. M. Carneiro INESC Porto, Telecommunications and Multimedia Unit "The universe is always one step beyond logic." -- Frank Herbert

2 1

C API equivalent to abc.ABCMeta.register?
by Daniel Stutzbach March 2, 2022

March 2, 2022

I'm the author of an extension module (blist) that provides a type that fits the MutableSequence API. Is there a canonical way for me to register the type as a MutableSequence from the C API? -- Daniel Stutzbach, Ph.D. President, Stutzbach Enterprises, LLC <http://stutzbachenterprises.com>

3 2

Software Development - Application Development - Game Development
by Gary Jan. 10, 2022

Jan. 10, 2022

You are receiving this email because we wish you to use our cost-effective IT services. We are a China based Custom Software Application Development provider. We offer full cycle custom software programming services, from product idea, software development to support and enhancement. We employ a large pool of software engineers coming from different backgrounds. We are able to balance product development efforts and project duration to your business needs. Core Services: Custom Application Development Custom Software Development iPhone Application Development iPad Application Development Mobile Application Development Android Mobile Application Development Soft Product Development Game Design & Development Game Testing & Quality Assurance Cloud Computing Solutions: CRM Software Application Development ERP Software Application Development Enterprise Portal Solution Sales Force Automation Solution Game Design & Development Knowledge Management Solution Workflow Management Solution SharePoint Development Services Microsoft Online Services Microsoft Azure Services Best regards, Gary ITVIATSA Software Development Contact: ibsoftware(a)yeah.net Pls send address to larryremove123(a)msn.com for remove

2 2

Mail l8st
by casperssidekick July 10, 2021

July 10, 2021

Sent from my Verizon, Samsung Galaxy smartphone

2 1

Module state access from slot methods: _PyType_GetModuleByDef
by Petr Viktorin Sept. 22, 2020

Sept. 22, 2020

Hello, Now that PEP 573 (Module State Access from C Extension Methods) is in, it's time to address the most important part missing from it: module state access from slot methods (like tp_init or nb_add). This has long history; see e.g. a thread I started in 2015: https://mail.python.org/pipermail/import-sig/2015-July/thread.html#1037 Most recently, a possible solution, an MRO walker, was one of the last things removed from PEP 573 before acceptance, because the proposed solution was broken: see https://mail.python.org/archives/list/python-dev@python.org/message/PFBAQWQ… Here's the removed text: > Slot methods > ------------ > > To allow access to `per-module state`_ from slot methods, an MRO walker > will be implemented:: > > PyTypeObject *PyType_DefiningTypeFromSlotFunc(PyTypeObject *type, > int slot, void *func) > > The walker will go through bases of heap-allocated ``type`` > and search for class that defines ``func`` at its ``slot``. > > The ``func`` does not need to be inherited by ``type`` (i.e. it may have been > overridden in a subclass). The only requirement for the walker to find the > defining class is that the defining class must be heap-allocated. > > On failure, exception is set and NULL is returned. and the last iteration of the implementation is here: https://github.com/Dormouse759/cpython/blob/41fe314c5124fa9434d56169868dfc0… A MRO walker is not efficient. The other viable options I'm aware of are: - putting a pointer in the class object (which would need reliable class-level storage, see https://mail.python.org/archives/list/capi-sig@python.org/thread/BWVQLET4L3… ) - solutions that aren't fully general, like context variables or putting a pointer in each instance (thorny because constructors/initializers are themselves slots) - "__typeslots__" mentioned in https://mail.python.org/pipermail/import-sig/2015-July/001035.html -- which are, IMO, quite invasive Given that, I believe that some kind of MRO walker is still the best immediate solution, mainly because it's easiest to implement. If done well, it can only use current public API (limiting the maintenance burden you'd have with something wired into the internals, and allowing it to be copied to 3rd party code to support older Python versions), and should be easy to replace with a more performant solution once that comes out. And if it's added, more modules could switch to the stable ABI and eschew process-global state, so we get more exposure and experience with the remaining problems. Now, the main problem with "PyType_DefiningTypeFromSlotFunc" is that it looked slot functions, which can be set from Python code, and doing so can easily introduce C-level failures. To solve this, my next thought is instead looking for something only available in C: the PyModuleDef. Here's an implementation of such an MRO walker, which I call "_PyType_GetModuleByDef": https://github.com/encukou/cpython/blob/98dd889575cf7d1688495983ba791e14894… My thinking is to start using this in 3.10 as internal API to get a feel for its strengths and weaknesses. It should be easy to rewrite the function to only use public API, and added to another project (like Cython). It should also be easy to make it public once we're sure it's the way forward. What are your thoughts?

5 8

PyLong_{As,From}ByteArray
by doodspav＠gmail.com June 23, 2020

June 23, 2020

Can we make `PyLong_AsByteArray(PyLongObject* v, unsigned char* bytes, size_t n, int little_endian, int is_signed);` and `PyObject * PyLong_FromByteArray(const unsigned char* bytes, size_t n, int little_endian, int is_signed);` available as publicly documented functions? They're already available internally, as can be seen here (https://github.com/python/cpython/blob/578c3955e0222ec7b3146197467fbb0fcfae…) so all that would be required (I think) would be exporting the symbols and updating the documentation (https://docs.python.org/3/c-api/long.html). The benefits would be that we could use this when we want to use fixed width integer types without having to check the width of fundamental integer types (so we could use `int32_t` rather than checking `sizeof(int)` or the corresponding macro). This would also be useful if someone wanted to use compiler extension fixed width types such as `__int128_t` (and the corresponding unsigned version).

2 1

gcc -DPy_NO_DEPRECATED_API option?
by Barry Scott June 19, 2020

June 19, 2020

I have been following the discussions on removing deprecated macros and functions from the C API. I checked my code, PyCXX and have a coupld of issues that is easy to fix. But I may have missed something. What would help me is to be able to build extensions with all the deprecated pieces of the C API removed. A bit like the way the limited C API works via the -DPy_LIMITED_API=0x03040000 option. What do you think? Barry

3 5

Re: [Python-Dev] Removal of _Py_ForgetReference from public header in 3.9 issue
by Petr Viktorin June 17, 2020

June 17, 2020

On 2020-06-14 22:10, cpython(a)nicwatson.org wrote: > Please excuse if this is the wrong mailing list. I couldn't find one for module maintainers. This is relevant to capi-sig(a)python.org; let's continue here. > I maintain an open source Python module in C. I'm trying to verify for the first time that the module still works with cpython 3.9. This module does *not* use the "limited" C API. > > In building my module against 3.9b3, I'm getting a missing declaration warning on _Py_ForgetReference. My module builds and passes test fine, this is just a compiler warning issue. What does the _Py_ForgetReference function do? The [documentation] says it's only for use in the interpereter core, so I'd assume it's . [documentation]: https://docs.python.org/3/c-api/refcounting.html > The change that caused this was made in: > > commit f58bd7c1693fe041f7296a5778d0a11287895648 > Author: Victor Stinner <vstinner(a)python.org> > Date: Wed Feb 5 13:12:19 2020 +0100 > > bpo-39542: Make PyObject_INIT() opaque in limited C API (GH-18363) > ... > > I definitely need the _Py_ForgetReference call for a particularly hairy error condition (https://github.com/jnwatson/py-lmdb/blob/master/lmdb/cpython.c#L888 if you're curious). In fact, my tests will seg fault if I don't have that call and trace refs is enabled. I can't follow the reasoning behind the code easily. Why do you use _Py_ForgetReference and PyObject_Del, instead of Py_DECREF(self)? > Should I put an #ifdef Py_TRACE_REFS around the call? Ignore it? What do you think is the proper resolution?

6 28

Instantiation of subclasses of ctypes
by Justin Capella June 15, 2020

June 15, 2020

The Structure interface of ctypes is intended to be subclassed yet the subclasses init and new aren't invoked. I think this was intended as an optimization but it is inconsistent and confusing. The only documentation about this behavior takes the form of a unit test, which may not have ever worked properly. There is an issue about this open here https://bugs.python.org/issue38860 I'm of the opinion it should not be "special", what does the community think?

1 0

Metaclasses and Limited API
by Sebastian Berg June 10, 2020

June 10, 2020

Hi, Since there is a discussion about the limited API and headers. A long time ago I asked about this on python-dev: It concerns me a bit that the limited API does not seem to support MetaClasses in PyType_FromSpec (or FromSpecWithBase). In particular, last time I checked, it seemed impossible to extend the type struct within such a metaclass [1]. There also seemed to be an issue with PyType_FromSpecWithBases allocating exactly a base type-object, instead of using `tp_alloc` as `PyType_Type.tp_new` would. This and how to efficiently cache Python objects in methods are the two main concerns I have right now for long-term full adoption of the limited API. (There are other difficulties, but for the ones I am aware, I at least have an idea how to they need to be addressed). I suppose it may actually be possible to hack around the FromSpec issues, although I am not sure right now. unless subclassing PyType_Type in C is for some reason different and very much discouraged? Cheers, Sebastian [1] And e.g. PySide seemed to "do" this, and got away with it, because there is an additional NULL at the end, as a termination for slots or so. Which for them is unused and they just happen to use.

3 5

How can we ship a header file, or even a dynamic library, to port existing C extension modules to the new incompatible Python C API?
by Victor Stinner June 9, 2020

June 9, 2020

Hi, As I wrote in a previous email, the C API of CPython is evolving slowly to hide more and more implementation details. Sadly, some changes break the backward compatibility. Sometimes, a new function must be used, except that the function doesn't exist in older Python versions. "Update on C API changes to hide implementation details" https://mail.python.org/archives/list/capi-sig@python.org/thread/HYFQHLGNCR… Would it be possible to ship a header file (a single ".h" file) and/or even a dynamic library to provide new functions to old Python versions? I'm not sure if it's possible to ship a header file as a package on PyPI and make it available for C compilers easily without monkey patching distutils. Another option is to encourage Linux distributions to package such header file to make it easy to install on the system. Another (worse?) option is to advise authors of each C extension module to vendor a copy of the header file in their project and update it from time to time when needed. At least in C, we can provide most "missing" functions on older Python versions as macros or static inline functions. Python 3.6 now requires a subset C99 with static inline functions. The situation is very similar to the Python 2 to Python 3 transition. Benjamin Peterson wrote the "six" which became quickly very popular. First, projects vendored a copy of the "six" module as a submodule of their project (src/project/six.py, imported as "project.six"). Then, slowly, the "six" module was shipped by Linux distributions or installed via "pip install six". Cython already does that internally. A few examples: https://github.com/cython/cython/blob/57b4bdeb88cc18f8e2f89fe9a7cf506d4ee0f… #if PY_VERSION_HEX < 0x030200A4 typedef long Py_hash_t; #define __Pyx_PyInt_FromHash_t PyInt_FromLong #define __Pyx_PyInt_AsHash_t __Pyx_PyIndex_AsHash_t #else #define __Pyx_PyInt_FromHash_t PyInt_FromSsize_t #define __Pyx_PyInt_AsHash_t __Pyx_PyIndex_AsSsize_t #endif #if CYTHON_COMPILING_IN_LIMITED_API #define __Pyx_PyThreadState_Current PyThreadState_Get() #elif !CYTHON_FAST_THREAD_STATE #define __Pyx_PyThreadState_Current PyThreadState_GET() #elif PY_VERSION_HEX >= 0x03060000 //#elif PY_VERSION_HEX >= 0x03050200 // Actually added in 3.5.2, but compiling against that does not guarantee that we get imported there. #define __Pyx_PyThreadState_Current _PyThreadState_UncheckedGet() #elif PY_VERSION_HEX >= 0x03000000 #define __Pyx_PyThreadState_Current PyThreadState_GET() #else #define __Pyx_PyThreadState_Current _PyThreadState_Current #endif #if PY_VERSION_HEX >= 0x030900A4 #define __Pyx_SET_REFCNT(obj, refcnt) Py_SET_REFCNT(obj, refcnt) #define __Pyx_SET_SIZE(obj, size) Py_SET_SIZE(obj, size) #else #define __Pyx_SET_REFCNT(obj, refcnt) Py_REFCNT(obj) = (refcnt) #define __Pyx_SET_SIZE(obj, size) Py_SIZE(obj) = (size) #endif Victor -- Night gathers, and now my watch begins. It shall not end until my death.

6 22