[Numpy-discussion] Segfault in PyArray_Item_XDECREF when using recarray object references titles

[Michael Droettboom]

I've run into a segfault that occurs in the array destructor with arrays 
containing object references with both names and titles.

When a field contains both and name and a title, the fields dictionary 
contains two entries for that field.  This means that the array item 
destructor (which iterates through the fields dictionary) will decref 
the pointed-to object twice.  If the first decref causes the object to 
be deleted, the second decref has the potential to segfault.

It seems the simplest patch is to set the object pointer to NULL after 
decref'ing, so the second decref will do nothing.  However, perhaps 
there is a way to avoid decref'ing twice in the first place.

I've attached a script that exercises the segfault, a gdb backtrace, and 
a patch.  You may need to adjust the number of rows until it is high 
enough to create a segfault on your system.

This is on:
RHEL4
Python 2.5.2
Numpy SVN r5497

Cheers,
Mike

> gdb python
GNU gdb Red Hat Linux (6.3.0.0-1.153.el4_6.2rh)
Copyright 2004 Free Software Foundation, Inc.
GDB is free software, covered by the GNU General Public License, and you are
welcome to change it and/or distribute copies of it under certain conditions.
Type "show copying" to see the conditions.
There is absolutely no warranty for GDB.  Type "show warranty" for details.
This GDB was configured as "i386-redhat-linux-gnu"...Using host libthread_db library "/lib/tls/libthread_db.so.1".

(gdb) run segfault.py
Starting program: /wonkabar/data1/usr/bin/python segfault.py
[Thread debugging using libthread_db enabled]
[New Thread -1208489312 (LWP 30028)]
len(dtype) = 1, len(dtype.fields) = 2
{'name': (dtype('object'), 0, 'title'), 'title': (dtype('object'), 0, 'title')}

Program received signal SIGSEGV, Segmentation fault.
[Switching to Thread -1208489312 (LWP 30028)]
0x0097285e in PyArray_Item_XDECREF (
    data=0xb7a3e780 "\uffff_\224\uffff `\214\uffff(`\214\uffff0`\214\uffff8`\214\uffff@`\214\uffffH`\214\uffffP`\214\uffffX`\214\uffff``\214\uffffh`\214\uffffp`\214\uffffx`\214\uffff\200`\214\uffff\210`\214\uffff\220`\214\uffff\230`\214\uffff\uffff`\214\uffff\uffff`\214\uffff\uffff`\214\uffff\uffff`\214\uffff\uffff`\214\uffff\uffff`\214\uffff\uffff`\214\uffff\uffff`\214\uffff\uffff`\214\uffff\uffff`\214\uffff\uffff`\214\uffff\uffff`\214\uffff",
    descr=0x9d4680) at numpy/core/src/arrayobject.c:198
198             Py_XDECREF(*temp);
(gdb) bt
#0  0x0097285e in PyArray_Item_XDECREF (
    data=0xb7a3e780 "\uffff_\224\uffff `\214\uffff(`\214\uffff0`\214\uffff8`\214\uffff@`\214\uffffH`\214\uffffP`\214\uffffX`\214\uffff``\214\uffffh`\214\uffffp`\214\uffffx`\214\uffff\200`\214\uffff\210`\214\uffff\220`\214\uffff\230`\214\uffff\uffff`\214\uffff\uffff`\214\uffff\uffff`\214\uffff\uffff`\214\uffff\uffff`\214\uffff\uffff`\214\uffff\uffff`\214\uffff\uffff`\214\uffff\uffff`\214\uffff\uffff`\214\uffff\uffff`\214\uffff\uffff`\214\uffff",
    descr=0x9d4680) at numpy/core/src/arrayobject.c:198
#1  0x00991bc7 in PyArray_XDECREF (mp=0xb7ae4f0c)
    at numpy/core/src/arrayobject.c:211
#2  0x009a579b in array_dealloc (self=0xb7ae4f0c)
    at numpy/core/src/arrayobject.c:2089
#3  0x0809781f in subtype_dealloc (self=0xb7ae4f0c) at Objects/typeobject.c:709
#4  0x08082a02 in PyDict_SetItem (op=0xb7f56acc, key=0xb7ea7d80,
    value=0x81379c0) at Objects/dictobject.c:416
#5  0x08085a1e in _PyModule_Clear (m=0xb7f3e0ec) at Objects/moduleobject.c:136
#6  0x080d7138 in PyImport_Cleanup () at Python/import.c:439
#7  0x080e4343 in Py_Finalize () at Python/pythonrun.c:399
#8  0x08056633 in Py_Main (argc=1, argv=0xbff1ca24) at Modules/main.c:545
#9  0x08056323 in main (argc=2, argv=0xbff1ca24) at ./Modules/python.c:23


-- 
Michael Droettboom
Science Software Branch
Operations and Engineering Division
Space Telescope Science Institute
Operated by AURA for NASA

-------------- next part --------------
An embedded and charset-unspecified text was scrubbed...
Name: arrayobject.c.diff
URL: <http://mail.python.org/pipermail/numpy-discussion/attachments/20080722/38f3cc9e/attachment.ksh>
-------------- next part --------------
An embedded and charset-unspecified text was scrubbed...
Name: segfault.py
URL: <http://mail.python.org/pipermail/numpy-discussion/attachments/20080722/38f3cc9e/attachment-0001.ksh>