Canonical test_XXX.py - nope (was:Re: [Python-Dev] Re: Call for reviewer!)

[Tim Peters]

[David Goodger]
> Glad to contribute. You'll find a regression test module for the current
> getopt.py as revised patch #101110. I based it on some existing
> Lib/test/ modules, but haven't found the canonical example or instruction
> set. Is there one?

[Trent Mick]
> I don't really think there is. Kind of folk lore. There are some good
> examples to follow in the existing test suite. Skip Montanaro
> wrote a README for writing tests and using the test suite
> (.../dist/src/Lib/test/README).
>
> Really, the testing framework is extremely simple. Which is one of its
> benefits. There is not a whole lot of depth that one has not
> grokked just by writing one test_XXX.py.

What he said.  The README explains it well, and I think the only thing you
(David) missed in your patch was the need to generate the "expected output"
file via running regrtest once with -g on the new test case.

I'd add one thing:  people use "assert" *way* too much in the test suite.
It's usually much better to just print what you got, and rely on regrtest's
output-comparison to complain if what you get isn't what you expected.  The
primary reason for this is that asserts "vanish" when Python is run
using -O, so running regrtest in -O mode simply doesn't test *anything*
caught by an assert.

A compromise is to do both:

    print what_i_expected, what_i_got
    assert what_i_expected == what_i_got

In Python 3000, I expect we'll introduce a new binary infix operator

    !!!

so that

    print x !!! y

both prints x and y and asserts that they're equal <wink>.