[Python-Dev] A Subtle Bug in Class Initializations

[Benjamin Peterson]

This would be a good thing to fix. The only hard part is dealing with thirdparty extensions.

Note we also have been working around this problem by putting PyType_Ready calls in various generic code paths of the interpreter when an uninitialized type passes through.

On Mon, Aug 6, 2018, at 11:02, Eddie Elizondo wrote:
> Solution:
> Here are my proposed solutions in order from less controversial to most 
> controversial. Note that all of them I already tried in a local branch 
> and are working:
> 
> 1) Initialize all uninitialized types.
> 
> Example: 
> https://github.com/eduardo-elizondo/cpython/commit/bc53db3cf4e5a6923b0b1afa6181305553faf173

That fixes CPython but what about thirdparty extensions?

> 
> 2) Move all PyVarObject_HEAD_INIT to NULL except PyType_Type, 
> PyBaseObject_Type and the booleans.
> 
> 3) Special case the initialization of PyType_Type and PyBaseObject_Type 
> within PyType_Ready to now make all calls to PyVarObject_HEAD_INIT use 
> NULL. To enable this a small change within PyType_Ready is needed to 
> initialize PyType_Type PyBaseObject:
> if (base == NULL) {
>     Py_TYPE(&PyType_Type) = &PyType_Type;
>     Py_TYPE(type) = &PyType_Type;
> }
> 
> Also, the booleans have to be fully initialized without calling 
> PyVarObject_HEAD_INIT. I propose:
> 
> struct _longobject _Py_FalseStruct = {
>     PyObject_HEAD_INIT(&PyBool_Type), 0, { 0 }
> };
> 
> This will clean-up the entire codebase of this anti-pattern.
> 
> Example: 
> https://github.com/eduardo-elizondo/cpython/commit/542fd79e4279c64c077c127b175a8d856d3c5f0b
> 
> 4) Modify PyVarObject_HEAD_INIT to ignore the input and initialize to 
> NULL and 0.
> In order to prevent this antipattern within extension code as well, we 
> should make PyVarObject_HEAD_INIT ignore the inputs and just set the 
> value to NULL.
> 
> #define PyVarObject_HEAD_INIT(type, size)       \
>     { PyObject_HEAD_INIT(NULL) 0 },
> 
> This will prevent external code to have a semi-initialized type that is 
> not initialized through PyType_Ready.

When does that fail, though? The first time someone tries to do anything with the type?

> 
> 5) Finally, I would go even further and suggest making 
> PyVarObject_HEAD_INIT argumentless.
> 
> #define PyVarObject_HEAD_INIT       \
>     { PyObject_HEAD_INIT(NULL) 0 },
> 
> However, this breaks backward compatibility. That being said, this will 
> make extension maintainers aware of this antipattern.