[Numpy-discussion] Segfault in PyArray_Item_XDECREF when using recarray object references titles

Michael Droettboom mdroe at stsci.edu
Wed Aug 6 11:01:48 EDT 2008


I've filed a bug, with a patch to address all these issues, here:

http://scipy.org/scipy/numpy/ticket/877

Cheers,
Mike

Michael Droettboom wrote:
> I also noticed that the inverse operation, PyArray_Item_INCREF has the 
> potential to leak memory as it will doubly-increment each object in the 
> array.  The solution there probably isn't quite as clean, since we can't 
> just mark the pointer.  It will have to somehow avoid incref'ing the 
> objects twice when iterating through the fields dictionary.
>
> Cheers,
> Mike
>
> Michael Droettboom wrote:
>   
>> I've run into a segfault that occurs in the array destructor with 
>> arrays containing object references with both names and titles.
>>
>> When a field contains both and name and a title, the fields dictionary 
>> contains two entries for that field.  This means that the array item 
>> destructor (which iterates through the fields dictionary) will decref 
>> the pointed-to object twice.  If the first decref causes the object to 
>> be deleted, the second decref has the potential to segfault.
>>
>> It seems the simplest patch is to set the object pointer to NULL after 
>> decref'ing, so the second decref will do nothing.  However, perhaps 
>> there is a way to avoid decref'ing twice in the first place.
>>
>> I've attached a script that exercises the segfault, a gdb backtrace, 
>> and a patch.  You may need to adjust the number of rows until it is 
>> high enough to create a segfault on your system.
>>
>> This is on:
>> RHEL4
>> Python 2.5.2
>> Numpy SVN r5497
>>
>> Cheers,
>> Mike
>>
>>     
>>> gdb python
>>>       
>> GNU gdb Red Hat Linux (6.3.0.0-1.153.el4_6.2rh)
>> Copyright 2004 Free Software Foundation, Inc.
>> GDB is free software, covered by the GNU General Public License, and 
>> you are
>> welcome to change it and/or distribute copies of it under certain 
>> conditions.
>> Type "show copying" to see the conditions.
>> There is absolutely no warranty for GDB.  Type "show warranty" for 
>> details.
>> This GDB was configured as "i386-redhat-linux-gnu"...Using host 
>> libthread_db library "/lib/tls/libthread_db.so.1".
>>
>> (gdb) run segfault.py
>> Starting program: /wonkabar/data1/usr/bin/python segfault.py
>> [Thread debugging using libthread_db enabled]
>> [New Thread -1208489312 (LWP 30028)]
>> len(dtype) = 1, len(dtype.fields) = 2
>> {'name': (dtype('object'), 0, 'title'), 'title': (dtype('object'), 0, 
>> 'title')}
>>
>> Program received signal SIGSEGV, Segmentation fault.
>> [Switching to Thread -1208489312 (LWP 30028)]
>> 0x0097285e in PyArray_Item_XDECREF (
>>    data=0xb7a3e780 "\uffff_\224\uffff 
>> `\214\uffff(`\214\uffff0`\214\uffff8`\214\uffff@`\214\uffffH`\214\uffffP`\214\uffffX`\214\uffff``\214\uffffh`\214\uffffp`\214\uffffx`\214\uffff\200`\214\uffff\210`\214\uffff\220`\214\uffff\230`\214\uffff\uffff`\214\uffff\uffff`\214\uffff\uffff`\214\uffff\uffff`\214\uffff\uffff`\214\uffff\uffff`\214\uffff\uffff`\214\uffff\uffff`\214\uffff\uffff`\214\uffff\uffff`\214\uffff\uffff`\214\uffff\uffff`\214\uffff", 
>>
>>    descr=0x9d4680) at numpy/core/src/arrayobject.c:198
>> 198             Py_XDECREF(*temp);
>> (gdb) bt
>> #0  0x0097285e in PyArray_Item_XDECREF (
>>    data=0xb7a3e780 "\uffff_\224\uffff 
>> `\214\uffff(`\214\uffff0`\214\uffff8`\214\uffff@`\214\uffffH`\214\uffffP`\214\uffffX`\214\uffff``\214\uffffh`\214\uffffp`\214\uffffx`\214\uffff\200`\214\uffff\210`\214\uffff\220`\214\uffff\230`\214\uffff\uffff`\214\uffff\uffff`\214\uffff\uffff`\214\uffff\uffff`\214\uffff\uffff`\214\uffff\uffff`\214\uffff\uffff`\214\uffff\uffff`\214\uffff\uffff`\214\uffff\uffff`\214\uffff\uffff`\214\uffff\uffff`\214\uffff", 
>>
>>    descr=0x9d4680) at numpy/core/src/arrayobject.c:198
>> #1  0x00991bc7 in PyArray_XDECREF (mp=0xb7ae4f0c)
>>    at numpy/core/src/arrayobject.c:211
>> #2  0x009a579b in array_dealloc (self=0xb7ae4f0c)
>>    at numpy/core/src/arrayobject.c:2089
>> #3  0x0809781f in subtype_dealloc (self=0xb7ae4f0c) at 
>> Objects/typeobject.c:709
>> #4  0x08082a02 in PyDict_SetItem (op=0xb7f56acc, key=0xb7ea7d80,
>>    value=0x81379c0) at Objects/dictobject.c:416
>> #5  0x08085a1e in _PyModule_Clear (m=0xb7f3e0ec) at 
>> Objects/moduleobject.c:136
>> #6  0x080d7138 in PyImport_Cleanup () at Python/import.c:439
>> #7  0x080e4343 in Py_Finalize () at Python/pythonrun.c:399
>> #8  0x08056633 in Py_Main (argc=1, argv=0xbff1ca24) at Modules/main.c:545
>> #9  0x08056323 in main (argc=2, argv=0xbff1ca24) at ./Modules/python.c:23
>>
>>
>> ------------------------------------------------------------------------
>>
>> _______________________________________________
>> Numpy-discussion mailing list
>> Numpy-discussion at scipy.org
>> http://projects.scipy.org/mailman/listinfo/numpy-discussion
>>     
>
>   

-- 
Michael Droettboom
Science Software Branch
Operations and Engineering Division
Space Telescope Science Institute
Operated by AURA for NASA




More information about the NumPy-Discussion mailing list