On Wed, Sep 26, 2018 at 04:03:16PM +0100, Rhodri James wrote:
> Assuming that you
> aren't doing some kind of wide-ranging static analysis (which doesn't
> seem to be what we're talking about), all that the contracts have bought
> you is the assurance that *this* invocation of the function with *these*
> parameters giving *this* result is what you expected. It does not say
> anything about the reliability of the function in general.
This is virtually the complete opposite of what contracts give us. What
you are describing is the problem with *unit testing*, not contracts.
I think Steven's is backwards in its own way.
- Contracts test the space of arguments actually used during testing period (or during initial production if the performance hit is acceptable).
- Unit tests test the space of arguments thought of by the developers.
A priori, either one of those can cover cases not addressed by the other. If unit tests use the hypothesis library or similar approaches, unit tests might very well examine arguments unlikely to be encountered in real-world (or test phase) use... these are nonetheless edge cases that are important to assure correct behavior on ("correct" can mean various things, of course: exceptions, recovery, default values whatever).
In contrast, contracts might well find arguments that the developers of unit tests had not thought of. I tend to think someone sitting down trying to think of edge cases is going to be able to write more thorough tests than the accident of "what did we see during this run" ... but it could go either way.
Of course... my own approach to this concern would more likely be to use a logging decorator rather than a DbC one. Then collect those logs that show all the arguments that were passed to a given function during a testing period, and roll those back into the unit tests. My approach is a bit more manual work, but also more flexible and more powerful.
- Half of the checks are very far away, in a separate file, assuming
I even remembered or bothered to write the test.
To me, this is the GREATEST VIRTUE of unit tests over DbC. It puts the tests far away where they don't distract from reading and understanding the function itself. I rarely want my checks proximate since I wear a very different hat when thinking about checks than when writing functionality (ideally, a lot of the time, I wear the unit test hat *before* I write the implementation; TDD is usually good practice).
- The post-conditions aren't checked unless I run my test suite, and
then they only check the canned input in the test suite.
Yes, this is a great advantage of unit tests. No cost until you explicitly run them.
- No class invariants.
- Inheritance is not handled correctly.
These are true. Also they are things I care very little about.