Does automatic golden master unittest generation exist/or is it feasible?
steve at pearwood.info
Tue Apr 4 23:43:51 EDT 2017
On Tue, 04 Apr 2017 19:40:03 -0700, fleshw wrote:
> People told me that there's a similar concept called "Golden Master
> Testing" where it keeps a "golden record" and runs test case on it. So
> it looks like I'm trying to automate golden master testing.
> So my question is this:
> 1. Is this a concept already explored?
Gold master testing is not a substitute for unit tests. The purpose of
gold master testing is different: not the ensure that the code is
correct, but to ensure that the behaviour *doesn't change*. Or at least
not without a human review to ensure the change is desired.
You would use gold master testing when you have a legacy code base that
needs refactoring. You don't care so much about fixing bugs (either there
aren't any known bugs, or you have to keep them for backwards
compatibility, or simply there's no budget for fixing them yet) but you
care about keeping the behaviour unchanged as you refactor the code.
If that describes your situation, then gold master testing is worth
pursuing. If not, then forget about it. Just write some unit tests.
I like small functions that take some arguments, do one thing, and return
a result without any side-effects. They are very easy to test using
doctests. Doctests make good documentation too, so you can kill two birds
with one stone. Write doctests.
For anything too complicated for a doctest, or for extensive and detailed
functional tests that exercise all the corners of your function, write
unit tests. The `unittest` module can run your doctests too,
automatically turning your doctests into a test case.
If your code base is well-designed, with lots of encapsulation, then
writing unit tests are easy.
If its not... well, you have my sympathy.
> 2. Is there already a framework for doing this?
This came up in googling. I make no comment about its maturity,
functionality, bugginess or overall quality, as I have never used it.
More information about the Python-list