[Python-Dev] Unit testing (again)

[Tim Peters]

[/F]
> But having looked everything over one more time, and having ported
> a small test suite to doctest.py, I'm now -0 on adding more test
> frameworks to 2.1.  If it's good enough for tim...

I'm not sure that it is, but I have yet to make time to look at the others.

It's no secret that I love doctest, and, indeed, in 20+ years of industry
pain, it's the only testing approach I didn't drop ASAP.  I still use it for
all my stuff, and very happily.

But!  I don't do anything with the web or GUIs etc -- I'm an algorithms guy.
Most of the stuff I work with has clearly defined input->output
relationships, and capturing an interactive session is simply perfect both
for documenting and testing such stuff.

It's also the case that I weight the "doc" part of "doctest" more heavily
than the "test" part, and when Peter or Guido say that, e.g., the reliance
on exact output match is "a problem", I couldn't disagree more strongly.
It's obvious to Guido that dict output may come in any order, but a doc
*reader* in a hurry is at best uneasy when documented output doesn't match
actual output exactly.  That's not something I'll yield on.

[Andrew]
>    def testGetItemFails(self):
>        self.assertRaises(KeyError, self._getitemfail)
>
>    def _getitemfail(self):
>       return self.t[1]
>
> [vs]
>
> self.test_exc('self.t[1]', KeyError)

My brain doesn't grasp either of those at first glance.  But everyone who
has used Python a week grasps this:

class C:
    def __getitem__(self, i):
        """Return the i'th item.  i==1 raises KeyError.

        For example,
        >>> c = C()
        >>> c[0]
        0
        >>> c[1]
        Traceback (most recent call last):
          File "x.py", line 20, in ?
            c[1]
          File "x.py", line 14, in __getitem__
            raise KeyError("bad i: " + `i`)
        KeyError: bad i: 1
        >>> c[-1]
        -1
        """

        if i != 1:
            return i
        else:
            raise KeyError("bad i: " + `i`)

Cute:  Python changed the first line of its traceback output (used to say
"Traceback (innermost last):"), and current doctest wasn't expecting that.
For *doc* purposes, it's important that the examples match what Python
actually does, so that's a bug in doctest.