On Sun, Sep 30, 2018 at 02:50:28PM +1000, Chris Angelico wrote:
And yet all the examples I've seen have just been poor substitutes for unit tests. Can we get some examples that actually do a better job of selling contracts?
In no particular order... (1) Distance Unit tests are far away from the code you are looking at. Things which go together ought to be together, but typically unit tests are not just seperate from the thing they are testing, but in a completely different file. Contracts are right there, next to the thing they belong with, but without being mixed into the implementation of the method or function. (2) Self-documenting code Contracts are self-documenting code. Unit tests are not. Unit tests are full of boilerplate, creating instances, setting up test data, checking the result. Contracts simply cut to the chase and state the requirements and the promises made: the input must be a non-empty list the result will be a string starting with "Aardvark" as executable code. (3) The "Have you performed the *right* tests?" problem Unit tests are great, but they have a serious problem: they test ONLY the canned data you put in your test. For non-trivial functions, unit tests tell you nothing about the general behaviour of your function, only the specific behaviour with the given input: The key problem with testing is that a test (of any kind) that uses one particular set of inputs tells you nothing at all about the behaviour of the system or component when it is given a different set of inputs. http://thinkrelevance.com/blog/2013/11/26/better-than-unit-tests Contracts are one (partial) solution to this problem. If you have a unit test that does this: def test_spam(self): self.AssertEqual(spam(2).count("eggs"), 2) then it tests ONLY that spam(2) contains "eggs" twice, and that's it. It tells you nothing about whether spam(3) or spam(1028374527601) is correct. In fact, under Test Driven Development, it would be normal to write the test first, and then implement spam as a stub that does this: def spam(n): return "eggs eggs" proving my point that a passing test with one input doesn't mean the function is correct for another input. In contrast, the post-condition: def spam(n): ensure: result.count("eggs") == max(0, n) # implementation is tested on every invocation of spam (up to the point that you decide to disable post-condition checks). There is no need for separate tests for spam(2) and spam(3) and spam(1028374527601) unless you have some specific need for them. (Say, a regression test after fixing a particular bug.) (4) Inheritance Contracts are inherited, unit tests are not. (5) Unit tests and contracts are complementary, not alternatives Unit tests and contracts do overlap, and in the areas of overlap contracts are generally superior. But sometimes it is too hard to specify a contract in sufficient detail, and so unit tests are more appropriate. And for some especially simple functions, you can't specify the post-condition except by duplicating the implementation: def max(a, b): """Return the maximum of a and b.""" ensure: result == a if a >= b else b implementation: return a if a >= b else b In that case, a post-condition is a waste of time, and one should just unit test it. https://sebnozzi.github.io/362/contracts-replace-unit-tests/ Another way to think about it is that unit tests and contracts have different purposes. Pre-conditions and class invariants are a form of defensive programming that ensures that your prerequisites are met, that code is called with the correct parameters, etc. Unit tests are a way of doing spot tests that the code works with certain specified inputs. (6) Separation of concerns: function algorithm versus error checking Functions ought to validate their input, but doing so obfuscates the function implementation. Making that input validation a pre-condition separates the error checking and input validation from the algorithm proper, which helps make the code self-documenting. (7) You can't unit test loop invariants Some languages have support for testing loop invariants. You can't unit test loop invariants at all. https://en.wikipedia.org/wiki/Loop_invariant#Programming_language_support (8) Executable documentation Contracts are executable code that document what input is valid and what result is returned. Executable code is preferrable to dead comments because comments rot: "At Resolver we've found it useful to short-circuit any doubt and just refer to comments in code as 'lies'. " -- Michael Foord paraphrases Christian Muirhead on python-dev, 2009-03-22 Contracts can also be extracted by static tools. -- Steve