Embedding Python crash on PyTuple_New

[Arnaud Loonstra]

On 23-11-2021 13:07, Arnaud Loonstra wrote:
> Hi,
> 
> I've got Python embedded successfully in a program up until now as I'm 
> now running into weird GC related segfaults. I'm currently trying to 
> debug this but my understanding of CPython limits me here.
> 
> I'm creating a Tuple in C but it crashes on creating it after a while. 
> It doesn't make sense which makes me wonder something else must be 
> happening? Could be it just crashes here because the GC is cleaning up 
> stuff completely unrelated to the allocation of the new tuple? How can I 
> troubleshoot this?
> 
> I've got CPython compiled with  --with-valgrind --without-pymalloc 
> --with-pydebug
> 
> In C I'm creating a tuple with the following method:
> 
> static PyObject *
> s_py_zosc_tuple(pythonactor_t *self, zosc_t *oscmsg)
> {
>      assert(self);
>      assert(oscmsg);
>      char *format = zosc_format(oscmsg);
> 
>      PyObject *rettuple = PyTuple_New((Py_ssize_t) strlen(format) );
> 
> It segfaults here (frame 16) after 320 times (consistently)
> 
> 1   __GI_raise             raise.c          49   0x7ffff72c4e71
> 2   __GI_abort             abort.c          79   0x7ffff72ae536
> 3   fatal_error            pylifecycle.c    2183 0x7ffff7d84b4f
> 4   Py_FatalError          pylifecycle.c    2193 0x7ffff7d878b2
> 5   _PyObject_AssertFailed object.c         2200 0x7ffff7c93cf2
> 6   visit_decref           gcmodule.c       378  0x7ffff7dadfd5
> 7   tupletraverse          tupleobject.c    623  0x7ffff7ca3e81
> 8   subtract_refs          gcmodule.c       406  0x7ffff7dad340
> 9   collect                gcmodule.c       1054 0x7ffff7dae838
> 10  collect_with_callback  gcmodule.c       1240 0x7ffff7daf17b
> 11  collect_generations    gcmodule.c       1262 0x7ffff7daf3f6
> 12  _PyObject_GC_Alloc     gcmodule.c       1977 0x7ffff7daf4f2
> 13  _PyObject_GC_Malloc    gcmodule.c       1987 0x7ffff7dafebc
> 14  _PyObject_GC_NewVar    gcmodule.c       2016 0x7ffff7daffa5
> 15  PyTuple_New            tupleobject.c    118  0x7ffff7ca4da7
> 16  s_py_zosc_tuple        pythonactor.c    366  0x55555568cc82
> 17  pythonactor_socket     pythonactor.c    664  0x55555568dac7
> 18  pythonactor_handle_msg pythonactor.c    862  0x55555568e472
> 19  pythonactor_handler    pythonactor.c    828  0x55555568e2e2
> 20  sphactor_actor_run     sphactor_actor.c 855  0x5555558cb268
> ... <More>
> 
> Any pointer really appreciated.

I've found enabling PYTHONTRACEMALLOC=1 in the environment gives me a 
pointer to where to offending block was allocated.

I'm calling this method from C:

18    def handleSocket(self, addr, data, type, name, uuid):
19        if addr == "/pulse":
20            self.lampval += 1
21            return (addr, [0,0])

Modules/gcmodule.c:108: gc_decref: Assertion "gc_get_refs(g) > 0" 
failed: refcount is too small
Memory block allocated at (most recent call first):
   File "/home/arnaud/src/build-gazebosc-Desktop-Debug/bin/lampen.py", 
line 21

object address  : 0x7fffd81154c0
object refcount : 1
object type     : 0x7ffff7f3df20
object type name: list
object repr     : [117, 0]

Fatal Python error: _PyObject_AssertFailed
Python runtime state: initialized

Current thread 0x00007ffff2481640 (most recent call first):
   File "/home/arnaud/src/build-gazebosc-Desktop-Debug/bin/lampen.py", 
line 21 in handleSocket

Thread 0x00007ffff5d288c0 (most recent call first):
<no Python frame>

Now it clearly says the refcount is 1 so I'm puzzling what it means?

Rg,

Arnaud