[Python-Dev] Segmentation fault in collections.defaultdict

[Kevin Jacobs <jacobs@bioinformed.com>]

An aside before I report this bug:

_I_HATE_SOURCEFORGE_.  If it doesn't bloody accept anonymous bug reports
then it bloody well shouldn't let you type in a nice, detailed, well
through-out report and then toss it in the toilet when you hit Submit, and
also not allow one dive in after it by using the browser back button to
recover the text.   AAARRGGHH!!

Anyhow, back to our regularly scheduled bug report, which as we know should
have gone to Sourceforge, but isn't because I don't have time for more of
that particular form of masochism.  (If that doesn't sit well with you, then
feel free to ignore any scribblings below.)

Try this at home:
import collections
d=collections.defaultdict(int)
d.iterkeys().next()  # Seg fault
d.iteritems().next() # Seg fault
d.itervalues().next() # Fine and dandy

Python version:
Python 2.5a2 (trunk:46822M, Jun 10 2006, 13:14:15)
[GCC 4.0.2 20050901 (prerelease) (SUSE Linux)] on linux2

Discussion:
The segmentation fault only occurs where we'd expect StopIteration to be
raised.  ie, if the defaultdict has 3 elements, then only the fourth call
will result in a segmentation fault. Based on the following traceback, The
failure occurs at dictobject.c:dictiter_iternextkey:2204, which attempts to
INCREF the next non-existent key in the sequence.  Thus the current code
does not properly detect when it has run out of elements.

Not having an intimate knowledge of the internals of dictobject.c or the new
defaultdict implementation, the underlying problem is not immediately
apparent.  I wish I had more time to follow up on this, but my "random
poking around time" is already overdrawn and I must get back to less
enjoyable pursuits.


Traceback:
> gdb ./python
GNU gdb 6.3
Copyright 2004 Free Software Foundation, Inc.
GDB is free software, covered by the GNU General Public License, and you are
welcome to change it and/or distribute copies of it under certain
conditions.
Type "show copying" to see the conditions.
There is absolutely no warranty for GDB.  Type "show warranty" for details.
This GDB was configured as "x86_64-suse-linux"...Using host libthread_db
library "/lib64/tls/libthread_db.so.1".

(gdb) r t.py
Starting program: src/python-trunk/python t.py
[Thread debugging using libthread_db enabled]
[New Thread 46912504205344 (LWP 12545)]

Program received signal SIGSEGV, Segmentation fault.
[Switching to Thread 46912504205344 (LWP 12545)]
dictiter_iternextkey (di=0x2aaaaab980a0) at dictobject.c:2204
2204            Py_INCREF(key);
(gdb) back
#0  dictiter_iternextkey (di=0x2aaaaab980a0) at dictobject.c:2204
#1  0x0000000000460366 in wrap_next (self=<value optimized out>, args=<value
optimized out>, wrapped=<value optimized out>)
    at typeobject.c:3846
#2  0x0000000000415adc in PyObject_Call (func=0x2aaaaab90a50,
arg=0x2aaaaaac2050, kw=0x0) at abstract.c:1802
#3  0x0000000000481217 in PyEval_EvalFrameEx (f=0x6df8f0, throwflag=<value
optimized out>) at ceval.c:3776
#4  0x0000000000483a81 in PyEval_EvalCodeEx (co=0x2aaaaab7daf8,
globals=<value optimized out>, locals=<value optimized out>, args=0x0,
    argcount=0, kws=0x0, kwcount=0, defs=0x0, defcount=0, closure=0x0) at
ceval.c:2832
#5  0x0000000000483ce2 in PyEval_EvalCode (co=<value optimized out>,
globals=<value optimized out>, locals=<value optimized out>)
    at ceval.c:494
#6  0x00000000004a52f7 in PyRun_FileExFlags (fp=0x654010,
filename=0x7fffffc96546 "t.py", start=<value optimized out>,
globals=0x677070,
    locals=0x677070, closeit=1, flags=0x7fffffc95300) at pythonrun.c:1232
#7  0x00000000004a5612 in PyRun_SimpleFileExFlags (fp=<value optimized out>,
filename=0x7fffffc96546 "t.py", closeit=1,
    flags=0x7fffffc95300) at pythonrun.c:856
#8  0x0000000000411cbd in Py_Main (argc=<value optimized out>,
argv=0x7fffffc95418) at main.c:494
#9  0x00002aaaab0515aa in __libc_start_main () from /lib64/tls/libc.so.6
#10 0x00000000004112ba in _start () at start.S:113
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mail.python.org/pipermail/python-dev/attachments/20060610/21b91682/attachment.html