Mailman 3 September 2018 - capi-sig

[capi-sig]Open questions about borrowed references
by Victor Stinner 04 Sep '18

04 Sep '18

Hi, I'm working on the implementation of my new C API. Modifying macros to use function calls is straighforward, but it seems like borrowed references is a big issue. Doc/data/refcounts.dat helps to discover a few functions and macros, but when writing a test suite on the C API, I found many more functions: Py_TYPE(), Py_SETREF(), PyList_SetItem(), etc. https://pythoncapi.readthedocs.io/bad_api.html#borrowed-references I see borrowed references as a threat to optimizations, because we don't know how long the caller use the return objected. For example, if a list of numbers is specialized to store numbers as 32 bit numbers, PyList_GetItem() has to create a temporary PyObject, but when should we destroy this temporary PyObject? Reference counting used "correctly" fits well with such temporary objects. For Py_TYPE(), I like the straighfoward "Py_TYPE(obj)->tp_name" to get the name of the object type. But is it an issue here that Py_TYPE() returns a borrowed reference? I see an issue if the object is the only instance of a type and there is no other reference to the type. Example: void test(PyObject *obj) { PyTypeObject *type = Py_TYPE(obj); Py_DECREF(obj); printf("name: %s\n", type->tp_name); } It's posible that Py_DECREF() releases indirectly the type and so that type becomes a dangling pointer, no? What about Py_SETREF(): is it safe to use it? The user has to increment manually the reference counter. API doesn't force to use borrowed references, am I right? Victor

2 2

[capi-sig]Re: New pythoncapi GitHub organization and creation of a test suite for the Python C API
by Jeroen Demeyer 03 Sep '18

03 Sep '18

On 2018-09-03 20:50, Brett Cannon wrote: > The worry of unintended issues due to Cython > happening to use an C API that's actually under test seems like a > reasonable worry. We're using Python scripts and Python classes to run the standard Python test suite and nobody seems to find that a problem...

3 2

[capi-sig]Re: New pythoncapi GitHub organization and creation of a test suite for the Python C API
by Jeroen Demeyer 03 Sep '18

03 Sep '18

On 2018-08-31 10:31, Victor Stinner wrote: > I don't wante to write unit tests with Cython, cffi or anything else, > because I really need very thin control on everything. You don't need "very thin control on *everything*". You need very thin control on specific parts of the testsuite. And that's the nice thing about Cython: you can easily shift the amount of control that you take. You can write the usual unittest classes and test_FOO methods where you don't care much about control but write fine-tuned C code (either using Cython syntax or C syntax) where you do want control. Of course you may have other reasons to not want Cython...

6 6

[capi-sig]New pythoncapi GitHub organization and creation of a test suite for the Python C API
by Victor Stinner 03 Sep '18

03 Sep '18

Hi, To be able to work on a fork of CPython, I decided to create a new GitHub organization: https://github.com/pythoncapi/ I didn't want to work on my personal fork of CPython (vstinner/cpython). The organization has a single member: me, but I invited Eric Snow, Barry Warsaw and Pablo Galindo. Tell me if you want to join! I moved my documentation there, the source of https://pythoncapi.readthedocs.io/ website: https://github.com/pythoncapi/pythoncapi/ And I just forked CPython: https://github.com/pythoncapi/cpython I will experiment to implement the new C API in this fork, to not bother the upstream CPython. I started with a very simple change, replace PyTuple_GET_ITEM() macro with a function call. I also started to write unit tests: https://github.com/pythoncapi/cpython/blob/pythoncapi/capi/tests/test_tuple… I quickly experimented to write my own C unit test framework, but I'm not really satisfied of the "verbose" API. I'm looking at googletest and Check, does someone have an experience with a C unit test framework? I don't wante to write unit tests with Cython, cffi or anything else, because I really need very thin control on everything. Victor

4 5