[Python-ideas] Simplicity of C (was why is design-by-contracts not widely)

[Steven D'Aprano]

On Sun, Sep 30, 2018 at 06:33:08PM +1000, Chris Angelico wrote:
> On Sun, Sep 30, 2018 at 6:03 PM Steven D'Aprano <steve at pearwood.info> wrote:
> >
> > On Sun, Sep 30, 2018 at 02:50:28PM +1000, Chris Angelico wrote:
> >
> > > And yet all the examples I've seen have just been poor substitutes for
> > > unit tests. Can we get some examples that actually do a better job of
> > > selling contracts?
> >
> > In no particular order...
> >
> > (1) Distance
> 
> Don't doctests deal with this too? With the exact same downsides?

I'm not sure that I understand the question.

Yes, doctests are usually (but not always) close to the function too. 
That's a plus in favour of doctests. The downside is that if you want to 
show a metric tonne of examples, your docstring becomes huge. (Or move 
it out into a separate text file, which doctest can still run.)

Sometimes its a real struggle to write a good docstring that isn't huge.

But on the other hand, a good editor should allow you to collapse 
docstrings to a single line.


> > (2) Self-documenting code
> 
> Ditto

Sorry, are you asking if doctests are self-documenting code?

Since they aren't part of the implementation of the function, no they 
aren't.


> > (3) The "Have you performed the *right* tests?" problem
> 
> Great if your contracts can actually be perfectly defined.

Why do you think the contracts have to be perfectly defined?

I keep reading people accusing pro-Contracts people of being zealots who 
believe that Contracts are a magic bullet that solve every problem 100% 
perfectly. I don't know where people get this idea.

Contracts are a tool, like unit tests. Just as a single unit test is 
better than no unit tests, and two unit tests is better than one, same 
goes for contracts. If you have an imperfect (incomplete, not buggy!) 
contract, that's better than *no* contract at all.

If you would like to check five things, but can only specify three of 
them in a contract, that's better than not specifying any of them.


> Every time
> a weird case is mentioned, those advocating contracts (mainly Marko)
> give examples showing "hey, contracts can do that too", and they're
> just testing specifics.

Can you give an example?

Of course, contracts *can* be specific (at least in Eiffel), the syntax 
allows it. In pseudo-code:

    def spam(x, y):
        require:
            if x is None:
                y > 0
        # implementation goes here...

The precondition checks that if x is None, y must be greater than zero. 
On its own, that's not a very good contract since it tells us nothing 
about the case when x is not None, but (as per my comments above) its 
better than nothing.

How would you do this as a unit test? For starters, you need to move the 
precondition into the function implementation block, giving it an 
explicit raise:

    def spam(x, y):
        if x is None and y <= 0:
            raise SomeError
        # implementation continues...


Now you've given up any hope of disabling that specific check, short of 
editing the source code.

Then you have to make a unit test to check that it works the way you 
think it does:

     def test_if_x_is_none_y_cannot_be_negative_or_zero(self):
         # Naming things is hard...
         self.AssertRaise(SomeError, spam, None, -1)
         self.AssertRaise(SomeError, spam, None, 0)

Great! Now we know that if spam is passed None and -1, or None and 0, it 
will correctly fail. But the unit test doesn't give us any confidence 
that None and -9137 will fail, since we haven't tested that case.

Work-around: I often write unit tests like this:

     def test_if_x_is_none_y_cannot_be_negative_or_zero(self):
         for i in range(20):
             # I never know how many or how few values to check...
             n = random.randint(1, 1000)  # or which values...
             self.AssertRaise(SomeError, spam, None, -n)
         self.AssertRaise(SomeError, spam, None, 0)


As David points out in another post, it is true that Contracts also can 
only check the values that you actually pass to the application during 
the testing phase, but that is likely to be a larger set of values than 
those we put in the unit tests, which lets us gain confidence in the 
code more quickly and with less effort. Rather than write lots of 
explicit tests, we can just exercise the routine with realistic data and 
see if the contract checks fail.



[...]
> > (6) Separation of concerns: function algorithm versus error checking
> 
> Agreed, so long as you can define the contract in a way that isn't
> just duplicating the function's own body.

Sure. That's a case where contracts aren't really suitable and a few 
unit tests would be appropriate, but that's only likely to apply to 
post-conditions, not pre-conditions.


> Contracts are great for some situations, but I'm seeing a lot of cases
> where they're just plain not, yet advocates still say "use contracts,
> use contracts". Why?

For the same reason advocates say "Use tests, use tests" despite there 
being some cases where formal automated tests are too hard to use.

Use contracts for the 90% of the cases they help, don't dismiss them 
because of the 10% of cases they don't.

But honestly, if you prefer unit testing, that's great too. Nobody is 
trying to say that unit tests suck or that you shouldn't use them. This 
is about giving people the choice between contracts or unit tests or 
both. When I say "Use contracts", what I mean is "Use as many or as few 
contracts as makes sense for your application, according to your taste".

Right now, that choice is very restrictive, and I personally don't like 
the look and feel of existing contract libraries. I would give my right 
eye for Cobra-like syntax for contracts in Python:

http://cobra-language.com/trac/cobra/wiki/Contracts



-- 
Steve