I also noticed that the inverse operation, PyArray_Item_INCREF has the potential to leak memory as it will doubly-increment each object in the array. The solution there probably isn't quite as clean, since we can't just mark the pointer. It will have to somehow avoid incref'ing the objects twice when iterating through the fields dictionary. Cheers, Mike Michael Droettboom wrote:
I've run into a segfault that occurs in the array destructor with arrays containing object references with both names and titles.
When a field contains both and name and a title, the fields dictionary contains two entries for that field. This means that the array item destructor (which iterates through the fields dictionary) will decref the pointed-to object twice. If the first decref causes the object to be deleted, the second decref has the potential to segfault.
It seems the simplest patch is to set the object pointer to NULL after decref'ing, so the second decref will do nothing. However, perhaps there is a way to avoid decref'ing twice in the first place.
I've attached a script that exercises the segfault, a gdb backtrace, and a patch. You may need to adjust the number of rows until it is high enough to create a segfault on your system.
This is on: RHEL4 Python 2.5.2 Numpy SVN r5497
Cheers, Mike
gdb python GNU gdb Red Hat Linux (6.3.0.0-1.153.el4_6.2rh) Copyright 2004 Free Software Foundation, Inc. GDB is free software, covered by the GNU General Public License, and you are welcome to change it and/or distribute copies of it under certain conditions. Type "show copying" to see the conditions. There is absolutely no warranty for GDB. Type "show warranty" for details. This GDB was configured as "i386-redhat-linux-gnu"...Using host libthread_db library "/lib/tls/libthread_db.so.1".
(gdb) run segfault.py Starting program: /wonkabar/data1/usr/bin/python segfault.py [Thread debugging using libthread_db enabled] [New Thread -1208489312 (LWP 30028)] len(dtype) = 1, len(dtype.fields) = 2 {'name': (dtype('object'), 0, 'title'), 'title': (dtype('object'), 0, 'title')}
Program received signal SIGSEGV, Segmentation fault. [Switching to Thread -1208489312 (LWP 30028)] 0x0097285e in PyArray_Item_XDECREF ( data=0xb7a3e780 "\uffff_\224\uffff `\214\uffff(`\214\uffff0`\214\uffff8`\214\uffff@`\214\uffffH`\214\uffffP`\214\uffffX`\214\uffff``\214\uffffh`\214\uffffp`\214\uffffx`\214\uffff\200`\214\uffff\210`\214\uffff\220`\214\uffff\230`\214\uffff\uffff`\214\uffff\uffff`\214\uffff\uffff`\214\uffff\uffff`\214\uffff\uffff`\214\uffff\uffff`\214\uffff\uffff`\214\uffff\uffff`\214\uffff\uffff`\214\uffff\uffff`\214\uffff\uffff`\214\uffff\uffff`\214\uffff",
descr=0x9d4680) at numpy/core/src/arrayobject.c:198 198 Py_XDECREF(*temp); (gdb) bt #0 0x0097285e in PyArray_Item_XDECREF ( data=0xb7a3e780 "\uffff_\224\uffff `\214\uffff(`\214\uffff0`\214\uffff8`\214\uffff@`\214\uffffH`\214\uffffP`\214\uffffX`\214\uffff``\214\uffffh`\214\uffffp`\214\uffffx`\214\uffff\200`\214\uffff\210`\214\uffff\220`\214\uffff\230`\214\uffff\uffff`\214\uffff\uffff`\214\uffff\uffff`\214\uffff\uffff`\214\uffff\uffff`\214\uffff\uffff`\214\uffff\uffff`\214\uffff\uffff`\214\uffff\uffff`\214\uffff\uffff`\214\uffff\uffff`\214\uffff\uffff`\214\uffff",
descr=0x9d4680) at numpy/core/src/arrayobject.c:198 #1 0x00991bc7 in PyArray_XDECREF (mp=0xb7ae4f0c) at numpy/core/src/arrayobject.c:211 #2 0x009a579b in array_dealloc (self=0xb7ae4f0c) at numpy/core/src/arrayobject.c:2089 #3 0x0809781f in subtype_dealloc (self=0xb7ae4f0c) at Objects/typeobject.c:709 #4 0x08082a02 in PyDict_SetItem (op=0xb7f56acc, key=0xb7ea7d80, value=0x81379c0) at Objects/dictobject.c:416 #5 0x08085a1e in _PyModule_Clear (m=0xb7f3e0ec) at Objects/moduleobject.c:136 #6 0x080d7138 in PyImport_Cleanup () at Python/import.c:439 #7 0x080e4343 in Py_Finalize () at Python/pythonrun.c:399 #8 0x08056633 in Py_Main (argc=1, argv=0xbff1ca24) at Modules/main.c:545 #9 0x08056323 in main (argc=2, argv=0xbff1ca24) at ./Modules/python.c:23
------------------------------------------------------------------------
_______________________________________________ Numpy-discussion mailing list Numpy-discussion@scipy.org http://projects.scipy.org/mailman/listinfo/numpy-discussion
-- Michael Droettboom Science Software Branch Operations and Engineering Division Space Telescope Science Institute Operated by AURA for NASA