[Python-ideas] Why is design-by-contracts not widely adopted?

[David Mertz]

I'm not sure what to do with the repeated assurance that that various
things "are obvious."  It really is NOT the case that me, or Paul Moore, or
Hugh Fisher, or Greg Ewing are simply too simple minded to understand what
DbC is.

The off-putting evangelical quality around DbC is something like the
similar evangelical insistence that OOP solves all problems that one tended
to hear in the late-1980s to mid-1990s especially.  The fact that no one
can quite pin down just *what* this special quality of DbC is doesn't
help... the reality is that they really ARE NOT much different from
assertions, in either practice or theory.

Actually, the evangelical OOP thing connects with the evangelical DbC.  One
of the "advantages" of DbC is often stated as support for inheritance.  But
the truth is, I hardly ever use inheritance in my code, or at most very
shallow inheritance in ways where invariants are mostly not preserved.  My
use of OOP in Python is basically mixins plus magic methods... and no, it's
not because I don't understand OOP (I wrote, for example, some of the most
widely read papers on metaclass programming in Python, and designed and
wrote about making a—toy, admittedly—method resolution order and OOP system
for R; my frequent co-author wrote the canonical paper on C3 linearization,
in which I'm acknowledged for my edits, but am not an author).

I also wrote an article or two about DbC in Python in the early 2000s.
None of this is very new.

On Mon, Sep 24, 2018 at 3:47 AM Marko Ristin-Kaufmann <
marko.ristin at gmail.com> wrote:

> *Obvious benefits*
> You both seem to misconceive the contracts. The goal of the
> design-by-contract is not reduced to testing the correctness of the code,
> as I reiterated already a couple of times in the previous thread. The
> contracts document *formally* what the caller and the callee expect and
> need to satisfy when using a method, a function or a class. This is meant
> for a module that is used by multiple people which are not necessarily
> familiar with the code. They are *not *a niche. There are 150K projects
> on pypi.org. Each one of them would benefit if annotated with the
> contracts.
>

Greg Ewing, Chris Angelica, and Paul Moore make the point very well that
MOST code is simply not the sort of thing that is amenable to having
contracts.  What one needs to state is either Turing complete or trivially
reducible to type declarations.  Or the behavior is duck typed loosely
enough that it can do the right thing without being able to specify the
pre- and post-conditions more precisely than "do what this function does."
Very often, that looseness is simply NOT a problem, and is part of what
makes Python flexible and successful.

*Contracts are difficult to read.*
> David wrote:
>
>> To me, as I've said, DbC imposes a very large cost for both writers and
>> readers of code.
>>
>
> This is again something that eludes me and I would be really thankful if
> you could clarify. Please consider for an example, pypackagery (
> https://pypackagery.readthedocs.io/en/latest/packagery.html) and the
> documentation of its function resolve_initial_paths:
>

The documentation you show below is definitely beautiful  I guess that's
generated by your Sphinx enhancement, right?  There are similar systems for
pulling things out of docstrings that follow conventions, but there's a
small incremental value in making them live tests at the same time.

But reading the end-user documentation is not the *reading* I'm talking
about.  The reading I mean is looking at the actual source code files.
Stating all the invariants you want code to follow makes the definitions of
functions/methods longer and more cognitive effort to parse.  A ten line
function body is likely to be accompanied by 15 lines of invariants that
one can state about hopes for the function behavior.  That's not always
bad... but it's definitely not always good.

This is why unit tests are often much better, organizationally.  The
function definitions can remain relatively concise and clean because most
of the time when you are reading or modifying them you don't WANT to think
about those precise contracts.  Sure, maybe some tool support like folding
of contracts could make the burden less; but not less than putting them in
entirely separate files.

Most of the time, I'd like to look at the simplest expression of the code
possible.  Then on a different day, or with a different set of eyes, I can
look at the completely distinct file that has arbitrarily many unit tests
to check invariants I'd like functions to maintain.  Yes, the issues are a
little bit different in classes and nested hierarchies... but like I said,
I never write those, and tend not to like code that does.

> packagery.resolve_initial_paths(*initial_paths*)
>
> Resolve the initial paths of the dependency graph by recursively adding
> *.py files beneath given directories.
> Parameters:
>
> *initial_paths* (List[Path]) – initial paths as absolute paths
> Return type:
>
> List[Path]
> Returns:
>
> list of initial files (*i.e.* no directories)
> Requires:
>
>    - all(pth.is_absolute() for pth in initial_paths)
>
> Ensures:
>
>    - len(result) >= len(initial_paths) if initial_paths else result == []
>    - all(pth.is_absolute() for pth in result)
>    - all(pth.is_file() for pth in result)
>
> Again, this is a straw man.


> *Writing contracts is difficult.*
> David wrote:
>
>> To me, as I've said, DbC imposes a very large cost for both writers and
>> readers of code.
>>
>
> The effort of writing contracts include as of now:
> * include icontract (or any other design-by-contract library) to setup.py
> (or requirements.txt), one line one-off
> * include sphinx-icontract to docs/source/conf.py and
> docs/source/requirements.txt, two lines, one-off
> * write your contracts (usually one line per contract).
>

It's not much work to add `import icontract` of course.  But *writing
contracts* is a lot of work.  Usually they work best only for pure
functions, and only for ones that deal with rather complex data structures
(flat is better than nested).  In Python, you usually simply do not have to
think about the issues that contracts guard against.


> I think that ignorance plays a major role here. Many people have
> misconceptions about the design-by-contract. They just use 2) for more
> complex methods, or 3) for rather trivial methods. They are not aware that
> it's easy to use the contracts (1) and fear using them for non-rational
> reasons (*e.g., *habits).
>

Again, this evangelical spirit to "burn the heretics" really isn't going to
win folks over.  No one replying here is ignorant.  We all do not see any
"silver bullet" in DbC that you advocate.  It's a moderately useful style
that I wouldn't object to using if a codebase style guide demanded it.  But
it's rarely the tool I would reach for on my own.  I just find it easier to
understand and use a combination of assertions and unit tests.  Neither is
*exactly* the same thing as DbC, but it's pretty darn close in practice.

And no, my hesitance isn't because I don't understand boolean logic.  I've
also studied a bit of graduate logic and model theory in a long ago life.

Yours, David...

-- 
Keeping medicines from the bloodstreams of the sick; food
from the bellies of the hungry; books from the hands of the
uneducated; technology from the underdeveloped; and putting
advocates of freedom in prisons.  Intellectual property is
to the 21st century what the slave trade was to the 16th.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20180926/360c7a4f/attachment-0001.html>