[Python-Dev] An update on heap-allocated types

[Eddie Elizondo]

I’ll be adding a change to move the Py_INCREF of heap allocated types from PyType_GenericAlloc to PyObject_Init. You can follow the discussion and/or add comments to: https://bugs.python.org/issue35810. This change will make types created through PyType_FromSpec behave like classes in managed code. Thus, making CPython much safer.

Without this change, there are a couple of edge cases where the use of PyObject_{,GC}_New{,Var} does not correctly increase refcount. This leads to weird behavior, especially when migrating types from PyType_Ready to PyType_FromSpec. For example, consider a static type with tp_new = NULL and tp_dealloc = NULL. This type initializes instances through PyObject_New and never increases the type’s refcount. tp_dealloc will be a no-op since it's NULL and it's a static type. When this type is migrated to PyType_FromSpec, tp_dealloc will now inherit subtype_dealloc which decrefs the type. This leads to a crash.

For the vast majority of existing code this should not have a visible side effect. And, at worst, this will only cause some type to become immortal. I’ve added instructions in the “Porting to Python 3.8” to correctly account for this new incref along with examples. In general, there are only two cases that would require any modification:

  1.  If the type creates instances through PyObject_{,GC}_New{,Var} and the type manually increfs afterwards. The fix here is to remove that manual incref.
  2.  If the type has a custom tp_dealloc and it’s not decrefing the type. The fix here is that a custom tp_dealloc should ALWAYS decref the type.

Open to feedback/discussion so feel free to reply if you have any questions!

- Eddie Elizondo
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20190214/2736c245/attachment-0001.html>