[Python-Dev] Need help with test_mutants.py

[Guido van Rossum]

On 8/24/06, Tim Peters <tim.peters at gmail.com> wrote:
> [Guido]
> > There's a unit test "test_mutants" which I don't understand. If anyone
> > remembers what it's doing, please contact me -- after ripping out
> > dictionary ordering in Py3k,
>
> Is any form of dictionary comparison still supported, and, if so, what
> does "dict1 cmp_op dict2" mean now?

Only == and != are supported between dicts. All the work is done by
dict_equal().

> > it stops working.
>
> Traceback?

Not particularly interesting: without changes, the code immediately
bombs like this:

trying w/ lengths 90 90 .
Traceback (most recent call last):
  File "../Lib/test/test_mutants.py", line 152, in <module>
    test(100)
  File "../Lib/test/test_mutants.py", line 149, in test
    test_one(random.randrange(1, 100))
  File "../Lib/test/test_mutants.py", line 135, in test_one
    c = cmp(dict1, dict2)
TypeError: unorderable types: dict() > dict()

> > In particular, the code in test_one() requires changes, but I don't
> > know how... Please help!
>
> The keys and values of dict1 and dict2 are filled with objects of a
> user-defined class whose __cmp__ method randomly mutates dict1 and
> dict2.  dict1 and dict2 are initially forced to have the same number
> of elements, so in current Python:
>
>         c = cmp(dict1, dict2)
>
> triggers a world of pain, with the internal dict code doing fancy
> stuff comparing keys and values.  However, every key and value
> comparison /may/ mutate the dicts in arbitrary ways, so this is
> testing whether the dict comparison implementation blows up
> (segfaults, etc) when the dicts it's comparing mutate during
> comparison.
>
> If it's only ordering comparisons that have gone away for dicts, then,
> e.g., replacing
>
>     c = cmp(dict1, dict2)
>
> with
>
>     c = dict1 == dict2
>
> instead will still meet the test's intent.

I made that change, and changed class Horrid to define __eq__ instead
of __cmp__. Since dict_equal() only invokes PyObject_RichCompareBool()
with op==Py_EQ that should be all that's needed.

Now when I run it, it spits out an apaprently infinite number of dots.
Putting a print in that __eq__ method suggests it is never called. Do
you understand this?

If I change Horrid.__hash__ to always return 42, I get output like this:

trying w/ lengths 12 14
trying w/ lengths 48 52
trying w/ lengths 19 18
trying w/ lengths 10 9
trying w/ lengths 48 46
trying w/ lengths 58 55
trying w/ lengths 50 48
trying w/ lengths 45 50
trying w/ lengths 19 19 .
Traceback (most recent call last):
  File "../Lib/test/test_mutants.py", line 158, in <module>
    test(100)
  File "../Lib/test/test_mutants.py", line 155, in test
    test_one(random.randrange(1, 100))
  File "../Lib/test/test_mutants.py", line 141, in test_one
    c = dict1 == dict2
  File "../Lib/test/test_mutants.py", line 99, in __eq__
    return self.i == other.i
AttributeError: 'Horrid' object has no attribute 'i'
Segmentation fault

But it doesn't always end with a segfault -- most of the time, the
AttributeError is the last thing printed.

> No particular /result/ is expected.  The test passes if and only if
> Python doesn't crash.  When the test was introduced, it uncovered at
> least six distinct failure (crashing) modes across the first 20 times
> it was run, so it's well worth keeping around in some form.

Well, it looks like it did provoke another crash, so I'll play with it
some more. Thanks!

-- 
--Guido van Rossum (home page: http://www.python.org/~guido/)