[Python-Dev] Need help with test_mutants.py

[Guido van Rossum]

On 8/24/06, Guido van Rossum <guido at python.org> wrote:
> I made that change, and changed class Horrid to define __eq__ instead
> of __cmp__. Since dict_equal() only invokes PyObject_RichCompareBool()
> with op==Py_EQ that should be all that's needed.
>
> Now when I run it, it spits out an apaprently infinite number of dots.
> Putting a print in that __eq__ method suggests it is never called. Do
> you understand this?
>
> If I change Horrid.__hash__ to always return 42, I get output like this:
>
> trying w/ lengths 12 14
> trying w/ lengths 48 52
> trying w/ lengths 19 18
> trying w/ lengths 10 9
> trying w/ lengths 48 46
> trying w/ lengths 58 55
> trying w/ lengths 50 48
> trying w/ lengths 45 50
> trying w/ lengths 19 19 .
> Traceback (most recent call last):
>   File "../Lib/test/test_mutants.py", line 158, in <module>
>     test(100)
>   File "../Lib/test/test_mutants.py", line 155, in test
>     test_one(random.randrange(1, 100))
>   File "../Lib/test/test_mutants.py", line 141, in test_one
>     c = dict1 == dict2
>   File "../Lib/test/test_mutants.py", line 99, in __eq__
>     return self.i == other.i
> AttributeError: 'Horrid' object has no attribute 'i'
> Segmentation fault
>
> But it doesn't always end with a segfault -- most of the time, the
> AttributeError is the last thing printed.

As a follow up to this story line, this appeared to be a refcount bug
in dict_equal(). I believe the same bug is probably present in 2.5; it
isn't triggered by test_mutants.py because that only exercises
dict_compare, not dict_richcompare, and only the latter can call
dict_equal (when op==Py_EQ or op==Py_NE). The bug is here:

                        PyObject *key = a->ma_table[i].me_key;
                        /* temporarily bump aval's refcount to ensure
it stays
                           alive until we're done with it */
                        Py_INCREF(aval);
                        bval = PyDict_GetItem((PyObject *)b, key);

The fix is to put Py_INCREF(key) // Py_DECREF(key) around the call to
PyDict_GetItem(). Apparently what can happen is that the only
reference to the key is in the dict, and the evil mutation from the
comparison delete the object before the comparison is completely done
with it.

Should I attempt to reproduce this bug in 2.5 and fix it?

-- 
--Guido van Rossum (home page: http://www.python.org/~guido/)