[Python-ideas] Why is design-by-contracts not widely adopted?

[Stephen J. Turnbull]

Steven D'Aprano writes:

 > put (x: ELEMENT; key: STRING) is
 > 		-- Insert x so that it will be retrievable through key.
 > 	require
 > 		count <= capacity
 > 		not key.empty
 > 	do
 > 		... Some insertion algorithm ...
 > 	ensure
 > 		has (x)
 > 		item (key) = x 
 > 		count = old count + 1
 > 	end
 > 
 > Two pre-conditions, and three post-conditions. That's hardly
 > complex.

You can already do this:

    def put(self, x: Element, key: str) -> None:
        """Insert x so that it will be retrievable through key."""

        # CHECKING PRECONDITIONS
        _old_count = self.count
        assert self.count <= self.capacity,
        assert key

        # IMPLEMENTATION
        ... some assertion algorithm ...

        # CHECKING POSTCONDITIONS
        assert x in self
        assert self[key] == x
        assert self.count == _old_count

        return

I don't see a big advantage to having syntax, unless the syntax allows
you to do things like turn off "expensive" contracts only.  Granted,
you save a little bit of typing and eye movement (you can omit
"assert" and have syntax instead of an assignment for checking
postconditions dependent on initial state).

A document generator can look for the special comments (as with
encoding cookies), and suck in all the asserts following until a
non-assert line of code (or the next special comment).  The
assignments will need special handling, an additional special comment
or something.  With PEP 572, I think you could even do this:

    assert ((_old_count := self.count),)

to get the benefit of python -O here.

 > If I were writing this in Python, I'd write something like this:
 > 
 > def put(self, x, key):
 >     """Insert x so that it will be retrievable through key."""
 >     # Input checks are pre-conditions!
 >     if self.count > capacity:
 >         raise DatabaseFullError
 >     if not key:
 >         raise ValueError
 >     # .. Some insertion algorithm ...

But this is quite different, as I understand it.  Nothing I've seen in
the discussion so far suggests that a contract violation allows
raising differentiated exceptions, and it seems very unlikely from the
syntax in your example above.  I could easily see both of these errors
being retryable:

    for _ in range(3):
        try:
            db.put(x, key)
        except DatabaseFullError:
            db.resize(expansion_factor=1.5)
            db.put(x, key)
        except ValueError:
            db.put(x, alternative_key)
        
 > and then stick the post-conditions in a unit test, usually in a 
 > completely different file:

If you like the contract-writing style, why would you do either of
these instead of something like the code I wrote above?

 > So what's wrong with the status quo?
 > 
 > - The pre-condition checks are embedded right there in the 
 >   method implementation, mixing up the core algorithm with the 
 >   associated error checking.

You don't need syntax to separate them, you can use a convention, as I
did above.

 > - Which in turn makes it hard to distinguish the checks from
 >   the implementation, and impossible to do so automatically.

sed can do it, why can't we?

 > - Half of the checks are very far away, in a separate file,
 >   assuming I even remembered or bothered to write the test.

That was your choice.  There's nothing about the assert statement that
says you're not allowed to use it at the end of a definition.

 > - The post-conditions aren't checked unless I run my test suite, and
 >   then they only check the canned input in the test suite.

Ditto.

 > - The pre-conditions can't be easily disabled in production.

What's so hard about python -O?

 > - No class invariants.

Examples?

 > - Inheritance is not handled correctly.

Examples?  Mixins and classes with additional functionality should
work fine AFAICS.  I guess you'd have to write the contracts in each
subclass of an abstract class, which is definitely a minus for some of
the contracts.  But I don't see offhand why you would expect that the
full contract of a method of a parent class would typically make sense
without change for an overriding implementation, and might not make
sense for a class with restricted functionality.

 > The status quo is all so very ad-hoc and messy. Design By Contract 
 > syntax would allow (not force, allow!) us to add some structure to the 
 > code:
 > 
 > - requirements of the function
 > - the implementation of the function
 > - the promise made by the function

Possible already as far as I can see.  OK, you could have the compiler
enforce the structure to some extent, but the real problem IMO is
going to be like documentation and testing: programmers just won't do
it regardless of syntax to make it nice and compiler checkable.

 > Most of us already think about these as three separate things, and 
 > document them as such. Our code should reflect the structure of how we 
 > think about the code.

But what's the need for syntax?  How about the common (in this thread)
complaint that even as decorators, the contract is annoying, verbose,
and distracts the reader from understanding the code?  Note: I think
that, as with static typing, this could be mitigated by allowing
contracts to be optionally specified in a stub file.  As somebody
pointed out, it shouldn't be hard to write contract strippers and
contract folding in many editors.  (As always, we have to admit it's
very difficult to get people to change their editor!)

 > > In my experience this is very rarely true.  Most functions I 
 > > write are fairly short and easily grokked, even if they do complicated 
 > > things.  That's part of the skill of breaking a problem down, IMHO; if 
 > > the function is long and horrible-looking, I've already got it wrong and 
 > > no amount of protective scaffolding like DbC is going to help.
 > 
 > That's like saying that if a function is horrible-looking, then there's 
 > no point in writing tests for it.
 > 
 > I'm not saying that contracts are only for horrible functions, but 
 > horrible functions are the ones which probably benefit the most from 
 > specifying exactly what they promise to do, and checking on every 
 > invocation that they live up to that promise.

I think you're missing the point then: ISTM that the implicit claim
here is that the time spent writing contracts for a horrible function
would be better spent refactoring it.  As you mention in connection
with the Eiffel example, it's not easy to get all the relevant
contracts, and for a horrible function it's going to be hard to get
some of the ones you do write correct.

 > Python (the interpreter) does type checking. Any time you get a 
 > TypeError, that's a failed type check. And with type annotations, we can 
 > run a static type checker on our code too, which will catch many of 
 > these failures before we run the code.

But an important strength of contracts is that they are *always* run,
on any input you actually give the function.