Embedding Python crash on PyTuple_New

[Arnaud Loonstra]

On 23-11-2021 15:34, MRAB wrote:
> On 2021-11-23 12:07, Arnaud Loonstra wrote:
>> Hi,
>>
>> I've got Python embedded successfully in a program up until now as I'm
>> now running into weird GC related segfaults. I'm currently trying to
>> debug this but my understanding of CPython limits me here.
>>
>> I'm creating a Tuple in C but it crashes on creating it after a while.
>> It doesn't make sense which makes me wonder something else must be
>> happening? Could be it just crashes here because the GC is cleaning up
>> stuff completely unrelated to the allocation of the new tuple? How can I
>> troubleshoot this?
>>
>> I've got CPython compiled with  --with-valgrind --without-pymalloc
>> --with-pydebug
>>
>> In C I'm creating a tuple with the following method:
>>
>> static PyObject *
>> s_py_zosc_tuple(pythonactor_t *self, zosc_t *oscmsg)
>> {
>>       assert(self);
>>       assert(oscmsg);
>>       char *format = zosc_format(oscmsg);
>>
>>       PyObject *rettuple = PyTuple_New((Py_ssize_t) strlen(format) );
>>
>> It segfaults here (frame 16) after 320 times (consistently)
>>
>>
>> 1   __GI_raise             raise.c          49   0x7ffff72c4e71
>> 2   __GI_abort             abort.c          79   0x7ffff72ae536
>> 3   fatal_error            pylifecycle.c    2183 0x7ffff7d84b4f
>> 4   Py_FatalError          pylifecycle.c    2193 0x7ffff7d878b2
>> 5   _PyObject_AssertFailed object.c         2200 0x7ffff7c93cf2
>> 6   visit_decref           gcmodule.c       378  0x7ffff7dadfd5
>> 7   tupletraverse          tupleobject.c    623  0x7ffff7ca3e81
>> 8   subtract_refs          gcmodule.c       406  0x7ffff7dad340
>> 9   collect                gcmodule.c       1054 0x7ffff7dae838
>> 10  collect_with_callback  gcmodule.c       1240 0x7ffff7daf17b
>> 11  collect_generations    gcmodule.c       1262 0x7ffff7daf3f6
>> 12  _PyObject_GC_Alloc     gcmodule.c       1977 0x7ffff7daf4f2
>> 13  _PyObject_GC_Malloc    gcmodule.c       1987 0x7ffff7dafebc
>> 14  _PyObject_GC_NewVar    gcmodule.c       2016 0x7ffff7daffa5
>> 15  PyTuple_New            tupleobject.c    118  0x7ffff7ca4da7
>> 16  s_py_zosc_tuple        pythonactor.c    366  0x55555568cc82
>> 17  pythonactor_socket     pythonactor.c    664  0x55555568dac7
>> 18  pythonactor_handle_msg pythonactor.c    862  0x55555568e472
>> 19  pythonactor_handler    pythonactor.c    828  0x55555568e2e2
>> 20  sphactor_actor_run     sphactor_actor.c 855  0x5555558cb268
>> ... <More>
>>
>> Any pointer really appreciated.
>>
> You're creating a tuple that'll have the same number of members as the 
> length of a string? That looks strange to me.
> 
> How are you setting the tuple's members?

It's from a serialisation format called OSC. The string describes the 
type of bytes, every character is a type.

I'm creating the tuple as follows:

PyObject *rettuple = PyTuple_New((Py_ssize_t) strlen(format) );

Then I iterate the OSC message using the format string, (just showing 
handling an int (i))

     char type = '0';
     Py_ssize_t pos = 0;
     const void *data =  zosc_first(oscmsg, &type);
     while(data)
     {
         switch (type)
         {
         case('i'):
         {
             int32_t val = 9;
             int rc = zosc_pop_int32(oscmsg, &val);
             assert(rc == 0);
             PyObject *o = PyLong_FromLong((long)val);
             assert( o );
             rc = PyTuple_SetItem(rettuple, pos, o);
             assert(rc == 0);
             break;
         }

Full code is here:

https://github.com/hku-ect/gazebosc/blob/822452dfa27201db274d37ce09e835d98fe500b2/Actors/pythonactor.c#L360