[New-bugs-announce] [issue35160] PyObjects initialized with PyObject_New have uninitialized pointers to set to 0x1

[Serafeim Mellos]

New submission from Serafeim Mellos <fim at mellos.io>:

I looked but I couldn't find any mention of this either in the bug tracker or the docs so I assume this is not a desired behavior or a known proble but feel free to correct me if I'm wrong.

I have noticed a weird behavior in Python3.7 onwards with the way PyObject_New() works. I'm not sure what is the exact cause for the issue but even though the documentation mentions that it should initialize all fields described in the associated PyTypeObject, I think there's a problem that somehow causes uninitialized pointers to get initialized with strange values that break other functionality (eg Py_XDECREF).

In more detail, it seems like uninitialized pointers in PyObjects get initialized to 0x1 which can lead to SEGFAULTs when calling Py_XDECREF() on them since they are no longer valid pointers.

I have taken the example extension from the python docs (https://docs.python.org/3/extending/newtypes_tutorial.html) and modified lightly in order to surface the issue. You can find the sample extension in my github: https://github.com/fim/python_pyobjectnew_example

I have checked versions 3.5, 3,6 which are fine but 3.7.0, 3.7.1 and 3.8 seem to reproduce this issue.

You can see a simple transcript of how the issue manifests below:

$ python setup.py build
[...]
$ pip install . 
[...]
$ python -c 'import custom; custom.Custom()'
weird pointer has value of: 0x8ec480
deallocating weird pointer
$ python
Python 3.7.1 (default, Nov  3 2018, 09:33:27) 
[GCC 5.4.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import custom, gc
>>> custom.Custom()
weird pointer has value of: 0x1
<custom2.Custom object at 0x7f1a2b8e4ed0>
>>> gc.collect()
deallocating weird pointer
zsh: segmentation fault  python

----------
components: Extension Modules
messages: 329224
nosy: fim
priority: normal
severity: normal
status: open
title: PyObjects initialized with PyObject_New have uninitialized pointers to set to 0x1
type: crash
versions: Python 3.7, Python 3.8

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue35160>
_______________________________________