[Python-ideas] Why is design-by-contracts not widely adopted?
Marko Ristin-Kaufmann
marko.ristin at gmail.com
Mon Oct 8 02:10:07 EDT 2018
Hi,
I'd like to share a use pattern for contracts that might have got lost in
the discussion and which I personally grow to like more and more. I'm not
making any claims; this use pattern work for our team and I can't judge how
much of a benefit it would be to others.
Imagine there are two developers, Alice working on a package A, and Betty
working on a package B. Package A depends on package B.
Betty tested her package B with some test data D_B.
Alice tests her package A with some test data D_A. Now assume Betty did not
write any contracts for her package B. When Alice tests her package, she is
actually making an integration test. While she controls the inputs to B
from A, she can only observe the results from B, but not whether they are
correct by coincidence or B did its job correctly. Let's denote D'_B the
data that is given to B from her original test data D_A during Alice's
integration testing.
How can she test that package B gives the correct results on D'_B ? She
needs to manually record the data somehow (by dynamically mocking package B
and intercepting what gets passed from A to B?). She might fetch the tests
from the package B, copy/paste the test cases and append D'_B. Or she could
make a pull request and provide the extra test data directly to package B.
She needs to understand how Betty's unit tests work and see how D'_B fits
in there and what needs to be mocked.
All in all, not a trivial task if Alice is not familiar with the package B
and even less so if Alice and Betty don't work in the same organization.
Most of the time, Alice would not bother to test the dependencies on her
testing data D_A. She would assume that her dependencies work, and just
tests what comes out of them. If the results make sense, she would call it
a tick on her to-do list and move on with the next task.
Let's assume now that Betty wrote some contracts in her code. When Alice
runs the integration test of her package A, the contracts of B are
automatically verified on D'_B. While the contracts might not cover all the
cases that were covered in Betty's unit tests, they still cover some of
them. Alice can be a bit more confident that at least *something* was
checked on D'_B. Without the contracts, she would have checked *nothing* on
D'_B in most of her everyday programming.
You can consider writing contracts as a matter of economy in this story.
Betty might not need contracts for maintaining her package B -- she can
read her code, she can extend her test cases. However, you can see
contracts as a service to the package users, Alice in this case. Betty
helps Alice have some integration tests free-of-charge (free for Alice;
Betty of course pays the overhead of writing and maintaining the
contracts). Alice does not need to understand how B can be tested nor needs
to manually record data that needs to be passed to B. She merely runs her
test code and the checker library will do the testing of B on D'_B
automatically.
The utility of this service tends to grow exponentially in cases where
dependency trees grow exponentially as well. Imagine if we had Carol with
the package C, with the dependencies A -> B -> C. When Carol writes
contracts, she does a service not only to her direct users (Betty) but also
to the users of B (Alice). I don't see how Alice could practically cover
the case with dependencies A -> B -> C and test C with D'_C (*i.e. *test C
with the data coming from D_A) without the contracts unless she really
takes her time and gets familiar with dependencies of all here immediate
dependencies.
We found this pattern helpful in the team, especially during refactorings
where contracts provide an additional security net. We don't have time to
record and add tests of B for D'_B, and even less so of C for D'_C. The
contracts work thus as a good compromise for us (marginal overhead, but
better documentation and "free" integration tests rather than none).
Cheers,
Marko
On Sun, 30 Sep 2018 at 08:17, Marko Ristin-Kaufmann <marko.ristin at gmail.com>
wrote:
> Hi,
>
> I compiled a couple of issues on github to provide a more structured
> ground for discussions on icontract features:
> https://github.com/Parquery/icontract/issues (@David Maertz: I also
> included the issue with automatically generated __doc__ in case you are
> still interested in it).
>
> Cheers,
> Marko
>
> On Sat, 29 Sep 2018 at 17:27, Stephen J. Turnbull <
> turnbull.stephen.fw at u.tsukuba.ac.jp> wrote:
>
>> Steven D'Aprano writes:
>>
>> > put (x: ELEMENT; key: STRING) is
>> > -- Insert x so that it will be retrievable through key.
>> > require
>> > count <= capacity
>> > not key.empty
>> > do
>> > ... Some insertion algorithm ...
>> > ensure
>> > has (x)
>> > item (key) = x
>> > count = old count + 1
>> > end
>> >
>> > Two pre-conditions, and three post-conditions. That's hardly
>> > complex.
>>
>> You can already do this:
>>
>> def put(self, x: Element, key: str) -> None:
>> """Insert x so that it will be retrievable through key."""
>>
>> # CHECKING PRECONDITIONS
>> _old_count = self.count
>> assert self.count <= self.capacity,
>> assert key
>>
>> # IMPLEMENTATION
>> ... some assertion algorithm ...
>>
>> # CHECKING POSTCONDITIONS
>> assert x in self
>> assert self[key] == x
>> assert self.count == _old_count
>>
>> return
>>
>> I don't see a big advantage to having syntax, unless the syntax allows
>> you to do things like turn off "expensive" contracts only. Granted,
>> you save a little bit of typing and eye movement (you can omit
>> "assert" and have syntax instead of an assignment for checking
>> postconditions dependent on initial state).
>>
>> A document generator can look for the special comments (as with
>> encoding cookies), and suck in all the asserts following until a
>> non-assert line of code (or the next special comment). The
>> assignments will need special handling, an additional special comment
>> or something. With PEP 572, I think you could even do this:
>>
>> assert ((_old_count := self.count),)
>>
>> to get the benefit of python -O here.
>>
>> > If I were writing this in Python, I'd write something like this:
>> >
>> > def put(self, x, key):
>> > """Insert x so that it will be retrievable through key."""
>> > # Input checks are pre-conditions!
>> > if self.count > capacity:
>> > raise DatabaseFullError
>> > if not key:
>> > raise ValueError
>> > # .. Some insertion algorithm ...
>>
>> But this is quite different, as I understand it. Nothing I've seen in
>> the discussion so far suggests that a contract violation allows
>> raising differentiated exceptions, and it seems very unlikely from the
>> syntax in your example above. I could easily see both of these errors
>> being retryable:
>>
>> for _ in range(3):
>> try:
>> db.put(x, key)
>> except DatabaseFullError:
>> db.resize(expansion_factor=1.5)
>> db.put(x, key)
>> except ValueError:
>> db.put(x, alternative_key)
>>
>> > and then stick the post-conditions in a unit test, usually in a
>> > completely different file:
>>
>> If you like the contract-writing style, why would you do either of
>> these instead of something like the code I wrote above?
>>
>> > So what's wrong with the status quo?
>> >
>> > - The pre-condition checks are embedded right there in the
>> > method implementation, mixing up the core algorithm with the
>> > associated error checking.
>>
>> You don't need syntax to separate them, you can use a convention, as I
>> did above.
>>
>> > - Which in turn makes it hard to distinguish the checks from
>> > the implementation, and impossible to do so automatically.
>>
>> sed can do it, why can't we?
>>
>> > - Half of the checks are very far away, in a separate file,
>> > assuming I even remembered or bothered to write the test.
>>
>> That was your choice. There's nothing about the assert statement that
>> says you're not allowed to use it at the end of a definition.
>>
>> > - The post-conditions aren't checked unless I run my test suite, and
>> > then they only check the canned input in the test suite.
>>
>> Ditto.
>>
>> > - The pre-conditions can't be easily disabled in production.
>>
>> What's so hard about python -O?
>>
>> > - No class invariants.
>>
>> Examples?
>>
>> > - Inheritance is not handled correctly.
>>
>> Examples? Mixins and classes with additional functionality should
>> work fine AFAICS. I guess you'd have to write the contracts in each
>> subclass of an abstract class, which is definitely a minus for some of
>> the contracts. But I don't see offhand why you would expect that the
>> full contract of a method of a parent class would typically make sense
>> without change for an overriding implementation, and might not make
>> sense for a class with restricted functionality.
>>
>> > The status quo is all so very ad-hoc and messy. Design By Contract
>> > syntax would allow (not force, allow!) us to add some structure to the
>> > code:
>> >
>> > - requirements of the function
>> > - the implementation of the function
>> > - the promise made by the function
>>
>> Possible already as far as I can see. OK, you could have the compiler
>> enforce the structure to some extent, but the real problem IMO is
>> going to be like documentation and testing: programmers just won't do
>> it regardless of syntax to make it nice and compiler checkable.
>>
>> > Most of us already think about these as three separate things, and
>> > document them as such. Our code should reflect the structure of how we
>> > think about the code.
>>
>> But what's the need for syntax? How about the common (in this thread)
>> complaint that even as decorators, the contract is annoying, verbose,
>> and distracts the reader from understanding the code? Note: I think
>> that, as with static typing, this could be mitigated by allowing
>> contracts to be optionally specified in a stub file. As somebody
>> pointed out, it shouldn't be hard to write contract strippers and
>> contract folding in many editors. (As always, we have to admit it's
>> very difficult to get people to change their editor!)
>>
>> > > In my experience this is very rarely true. Most functions I
>> > > write are fairly short and easily grokked, even if they do
>> complicated
>> > > things. That's part of the skill of breaking a problem down, IMHO;
>> if
>> > > the function is long and horrible-looking, I've already got it wrong
>> and
>> > > no amount of protective scaffolding like DbC is going to help.
>> >
>> > That's like saying that if a function is horrible-looking, then
>> there's
>> > no point in writing tests for it.
>> >
>> > I'm not saying that contracts are only for horrible functions, but
>> > horrible functions are the ones which probably benefit the most from
>> > specifying exactly what they promise to do, and checking on every
>> > invocation that they live up to that promise.
>>
>> I think you're missing the point then: ISTM that the implicit claim
>> here is that the time spent writing contracts for a horrible function
>> would be better spent refactoring it. As you mention in connection
>> with the Eiffel example, it's not easy to get all the relevant
>> contracts, and for a horrible function it's going to be hard to get
>> some of the ones you do write correct.
>>
>> > Python (the interpreter) does type checking. Any time you get a
>> > TypeError, that's a failed type check. And with type annotations, we
>> can
>> > run a static type checker on our code too, which will catch many of
>> > these failures before we run the code.
>>
>> But an important strength of contracts is that they are *always* run,
>> on any input you actually give the function.
>>
>> _______________________________________________
>> Python-ideas mailing list
>> Python-ideas at python.org
>> https://mail.python.org/mailman/listinfo/python-ideas
>> Code of Conduct: http://python.org/psf/codeofconduct/
>>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20181008/0772bf05/attachment-0001.html>
More information about the Python-ideas
mailing list