On 2018-09-05 13:49, Petr Viktorin wrote:
> I'm not at a computer right now, but AFAIK if <class int>'s refcount
> drops to zero, we still try to deallocate it, and that causes an
> assertion and terminates the interpreter.
You're right but that's besides the original point. If code uses
refcounting correctly (regardless of using an "old" or "new" API), then
the class int will never be deallocated.
On 2018-09-05 15:20, Victor Stinner wrote:
> PyObject *obj = create_object();
> PyTypeObject *type = Py_TYPE(obj);
> Py_DECREF(obj);
> /* ... */
> /* type is now a dangling pointer */
>
> Do you consider that this example uses reference counting "correctly"?
No because Py_TYPE() is known to return a borrowed reference. So, if you
want to do anything with it besides immediately using it, you need to
Py_INCREF() it.
On 2018-09-05 14:51, Hugh Fisher wrote:
> As of Python 3.6 a static type cannot be a subclass of a dynamic
> type
This is true, but I don't really know why. This was added by Serhiy
Storchaka in https://bugs.python.org/issue22079 but without much
rationale. I'm guessing that heap types are too dynamic to reliably
subclass from, but I don't really know...
If you really have good reasons for having an extension type subclassing
a heap type, I would love to know them.
Jeroen.
On 2018-09-05 13:32, Petr Viktorin wrote:
> Only the type object, <class int>, would be immortal.
This is already the case. Extension types are statically allocated, so
they are effectively immortal.
On 2018-09-05 00:12, Victor Stinner wrote:
> Examples of Py_TYPE() usage. My issue is that replacing Py_TYPE() with
> a new function which increment the reference counter of the type would
> make the code less straightforward.
You misunderstood my question. Why would you want to replace Py_TYPE()
with a new function which increments the reference counter of the type?
It seems to me that you are proposing a solution in search of a problem...
On 2018-09-04 23:28, Victor Stinner wrote:
> Hi,
>
> Ah! I succeeded to get a dangling pointer to a deallocated type object
> using Py_TYPE(). The example is designed to trigger the bug, but it
> proves that it's important to hold a strong reference to the type when
> using Py_TYPE(). Maybe it's not an issue for the normal case in the
> wild, but at least my example provides that there is an issue in a
> corner case which waits to bite you.
What's your point really? It's known that Py_TYPE() returns a borrowed
reference, so your example code is buggy. It shouldn't surprise anyone
that borrowed references can crash CPython.
Hi,
I'm working on the implementation of my new C API. Modifying macros to
use function calls is straighforward, but it seems like borrowed
references is a big issue. Doc/data/refcounts.dat helps to discover a
few functions and macros, but when writing a test suite on the C API,
I found many more functions: Py_TYPE(), Py_SETREF(), PyList_SetItem(),
etc.
https://pythoncapi.readthedocs.io/bad_api.html#borrowed-references
I see borrowed references as a threat to optimizations, because we
don't know how long the caller use the return objected. For example,
if a list of numbers is specialized to store numbers as 32 bit
numbers, PyList_GetItem() has to create a temporary PyObject, but when
should we destroy this temporary PyObject? Reference counting used
"correctly" fits well with such temporary objects.
For Py_TYPE(), I like the straighfoward "Py_TYPE(obj)->tp_name" to get
the name of the object type. But is it an issue here that Py_TYPE()
returns a borrowed reference? I see an issue if the object is the only
instance of a type and there is no other reference to the type.
Example:
void test(PyObject *obj)
{
PyTypeObject *type = Py_TYPE(obj);
Py_DECREF(obj);
printf("name: %s\n", type->tp_name);
}
It's posible that Py_DECREF() releases indirectly the type and so that
type becomes a dangling pointer, no?
What about Py_SETREF(): is it safe to use it? The user has to
increment manually the reference counter. API doesn't force to use
borrowed references, am I right?
Victor
On 2018-09-03 20:50, Brett Cannon wrote:
> The worry of unintended issues due to Cython
> happening to use an C API that's actually under test seems like a
> reasonable worry.
We're using Python scripts and Python classes to run the standard Python
test suite and nobody seems to find that a problem...
On 2018-08-31 10:31, Victor Stinner wrote:
> I don't wante to write unit tests with Cython, cffi or anything else,
> because I really need very thin control on everything.
You don't need "very thin control on *everything*". You need very thin
control on specific parts of the testsuite. And that's the nice thing
about Cython: you can easily shift the amount of control that you take.
You can write the usual unittest classes and test_FOO methods where you
don't care much about control but write fine-tuned C code (either using
Cython syntax or C syntax) where you do want control.
Of course you may have other reasons to not want Cython...
Hi,
To be able to work on a fork of CPython, I decided to create a new
GitHub organization:
https://github.com/pythoncapi/
I didn't want to work on my personal fork of CPython
(vstinner/cpython). The organization has a single member: me, but I
invited Eric Snow, Barry Warsaw and Pablo Galindo. Tell me if you want
to join!
I moved my documentation there, the source of
https://pythoncapi.readthedocs.io/ website:
https://github.com/pythoncapi/pythoncapi/
And I just forked CPython:
https://github.com/pythoncapi/cpython
I will experiment to implement the new C API in this fork, to not
bother the upstream CPython.
I started with a very simple change, replace PyTuple_GET_ITEM() macro
with a function call. I also started to write unit tests:
https://github.com/pythoncapi/cpython/blob/pythoncapi/capi/tests/test_tuple…
I quickly experimented to write my own C unit test framework, but I'm
not really satisfied of the "verbose" API. I'm looking at googletest
and Check, does someone have an experience with a C unit test
framework?
I don't wante to write unit tests with Cython, cffi or anything else,
because I really need very thin control on everything.
Victor
