[Python-ideas] Simplicity of C (was why is design-by-contracts not widely)

[Steven D'Aprano]

On Sun, Sep 30, 2018 at 02:50:28PM +1000, Chris Angelico wrote:

> And yet all the examples I've seen have just been poor substitutes for
> unit tests. Can we get some examples that actually do a better job of
> selling contracts?

In no particular order...

(1) Distance 

Unit tests are far away from the code you are looking at. Things which 
go together ought to be together, but typically unit tests are not just 
seperate from the thing they are testing, but in a completely different 
file.

Contracts are right there, next to the thing they belong with, but 
without being mixed into the implementation of the method or function.


(2) Self-documenting code

Contracts are self-documenting code. Unit tests are not. Unit tests are 
full of boilerplate, creating instances, setting up test data, checking 
the result. Contracts simply cut to the chase and state the 
requirements and the promises made:

    the input must be a non-empty list
    the result will be a string starting with "Aardvark"

as executable code.


(3) The "Have you performed the *right* tests?" problem

Unit tests are great, but they have a serious problem: they test ONLY 
the canned data you put in your test. For non-trivial functions, unit 
tests tell you nothing about the general behaviour of your function, 
only the specific behaviour with the given input:

    The key problem with testing is that a test (of any kind) 
    that uses one particular set of inputs tells you nothing 
    at all about the behaviour of the system or component when 
    it is given a different set of inputs.

http://thinkrelevance.com/blog/2013/11/26/better-than-unit-tests

Contracts are one (partial) solution to this problem.

If you have a unit test that does this:

    def test_spam(self):
        self.AssertEqual(spam(2).count("eggs"), 2)

then it tests ONLY that spam(2) contains "eggs" twice, and that's it. It 
tells you nothing about whether spam(3) or spam(1028374527601) is 
correct. In fact, under Test Driven Development, it would be normal to 
write the test first, and then implement spam as a stub that does this:

    def spam(n):
        return "eggs eggs"

proving my point that a passing test with one input doesn't mean the 
function is correct for another input.

In contrast, the post-condition:

    def spam(n):
        ensure:
            result.count("eggs") == max(0, n)
        # implementation


is tested on every invocation of spam (up to the point that you decide 
to disable post-condition checks). There is no need for separate tests 
for spam(2) and spam(3) and spam(1028374527601) unless you have some 
specific need for them. (Say, a regression test after fixing a 
particular bug.)


(4) Inheritance

Contracts are inherited, unit tests are not.


(5) Unit tests and contracts are complementary, not alternatives

Unit tests and contracts do overlap, and in the areas of overlap 
contracts are generally superior. But sometimes it is too hard to 
specify a contract in sufficient detail, and so unit tests are more 
appropriate.

And for some especially simple functions, you can't specify the 
post-condition except by duplicating the implementation:

    def max(a, b):
        """Return the maximum of a and b."""
        ensure:
            result == a if a >= b else b
        implementation:
            return a if a >= b else b

In that case, a post-condition is a waste of time, and one should just 
unit test it.

https://sebnozzi.github.io/362/contracts-replace-unit-tests/

Another way to think about it is that unit tests and contracts have 
different purposes. Pre-conditions and class invariants are a form of 
defensive programming that ensures that your prerequisites are met, that 
code is called with the correct parameters, etc. Unit tests are a way of 
doing spot tests that the code works with certain specified inputs.


(6) Separation of concerns: function algorithm versus error checking

Functions ought to validate their input, but doing so obfuscates the 
function implementation. Making that input validation a pre-condition 
separates the error checking and input validation from the algorithm 
proper, which helps make the code self-documenting.


(7) You can't unit test loop invariants

Some languages have support for testing loop invariants. You can't unit 
test loop invariants at all.

https://en.wikipedia.org/wiki/Loop_invariant#Programming_language_support


(8) Executable documentation

Contracts are executable code that document what input is valid and what 
result is returned. Executable code is preferrable to dead comments 
because comments rot:

"At Resolver we've found it useful to short-circuit any doubt and just 
refer to comments in code as 'lies'. "
-- Michael Foord paraphrases Christian Muirhead on python-dev, 2009-03-22

Contracts can also be extracted by static tools.




-- 
Steve