Hi Chris and Paul,

Let me please answer your messages in one go as they are related.

Paul wrote:
For most single use (or infrequently used) functions, I'd argue that the trade-off *isn't* worth it.

Here's a quick example from the pip codebase:

# Retry every half second for up to 3 seconds
@retry(stop_max_delay=3000, wait_fixed=500)
def rmtree(dir, ignore_errors=False):
    shutil.rmtree(dir, ignore_errors=ignore_errors,
                  onerror=rmtree_errorhandler)

Absolutely, I agree. If it's a single-use or infrequently used function that hardly anybody uses, it's not worth it. Moreover, if some contracts are harder to figure out than the implementation, then they are probably in most cases not worth the effort, too.

Please mind that I said: any library would benefit from it, as in the users of any library on pipy would benefit from better, formal and more precise documentation. That doesn't imply that all the contracts need to be specified or that you have to specify the contract for every function, or that you omit the documentation altogether. Some contracts are simply too hard to get explicitly. Some are not meaningful even if you could write them down. Some you'd like to add, but run only at test time since too slow in production. Some run in production, but are not included in the documentation (e.g., to prevent the system to enter a bad state though it is not meaningful for the user to actually read that contract).

Since contracts are formally written, they can be verified. Human text can not. Specifying all the contracts is in most cases not meaningful. In my day-to-day programming, I specify contracts on the fly and they help me express formally to the next girl/guy who will use my code what to expect or what not. That does not mean that s/he can just skip the documentation or that contracts describe fully what the function does. They merely help -- help him/her what arguments and results are expected. That does not mean that I fully specified all the predicates on the arguments and the result. It's merely a help à la
* "Ah, this argument needs to be bigger than that argument!" 
* "Ah, the resulting dictionary is shorter than the input list!"
* "Ah, the result does not include the input list"
* "Ah, this function accepts only files (no directories) and relative paths!"
* "Ah, I put the bounding box outside of the image -- something went wrong!"
* "Ah, this method allows me to put the bounding box outside of the image and will fill all the outside pixels with black!" etc.

For example, if I have an object detector operating on a region-of-interest and returning bounding boxes of the objects, the postconditions will not be: "all the bounding boxes are cars", that would impossible. But the postcondition might check that all the bounding boxes are within the region-of-interest or slightly outside, but not completely outside etc.

Let's be careful not to make a straw-man here, i.e. to push DbC ad absurdum and then discard it that way.

Also, the implicit contracts code currently has are typically pretty loose. What you need to "figure out" is very general. Explicit contracts are typically demonstrated as being relatively strict, and figuring out and writing such contracts is more work than writing code with loose implicit contracts. Whether the trade-off of defining tight explicit contracts once vs inferring a loose implicit contract every time you call the function is worth it, depends on how often the function is called. For most single use (or  infrequently used)  functions, I'd argue that the trade-off *isn't* worth it.

I don't believe such an approach would ever be pragmatical, i.e. automatic versioning based on the contracts. It might hint it (as a warning: you changed a contract, but did not bump the version), but relying solely on this mechanism to get the versioning would imply that you specified all the contracts. Though Bertrand might have always envisioned it as the best state of the world, even he himself repeatedly said that it's better to specify rather 10% than 0% contracts and 20% rather than 10% contracts.

Chris wrote:
def fibber(n):
    return n if n < 2 else fibber(n-1) + fibber(n-2)
 
It depends who you are writing this function for -- for example, instead of the contract, why not just include the code implementation as the documentation? The only meaningful contract I could imagine:
@pre n >= 0

Everything else would be just repetition of the function. If you implemented some optimized and complex fibber_optimized() function, then your contracts would probably look like:
@pre n >= 0
@post (n < 2 ) or result == fibber_optimized(n-1) + fibber_optimized(n-2)
@post not (n < 2) or result == n

> Here is my try at the contracts. Assuming that there is a list of figures and that they have a property "displayed" and that "State.blocked" global variable refers to whether the interface is blocked or not::
> @post(lambda: all(figure.displayed for figure in figures)
> @post(lambda: not ipython.in_pylab_mode() or not State.blocked)
> @post(lambda: not interactive() or State.blocked)
> matplotlib.pyplot.show()
>

There is no such thing as "State.blocked". It blocks. The function
*does not return* until the figure has been displayed, and dismissed.
There's no way to recognize that inside the function's state.

Contracts are great when every function is 100% deterministic and can
maintain invariants and/or transform from one set of invariants to
another. Contracts are far less clear when the definitions are
muddier.

Sorry, it has been ages that I used matplotlib. I thought it meant "blocking" as in "the drawing thread blocks". Since blocking basically means halting the execution-- then the contracts can't help. They are checked before and after the function executes. They can not solve the halting problem. For that you need formal methods (and I doubt that formal methods would ever work for matplotlib). The contracts do not check what happens during the execution of the function. They are not meant to do that. Even the invariants of the class are checked before and after the call to public methods (and the private methods are allowed to break them!).

Please mind that this is actually not the argument pro/against the contracts -- you are discussing in this particular case a tool (different to DbC) which should test the behavior during the execution of a function.

Chirs wrote:
> However, contracts can be useful when testing the GUI -- often it is difficult to script the user behavior. What many people do is record a session and re-play it. If there is a bug, fix it. Then re-record. While writing unit tests for GUI is hard since GUI changes rapidly during development and scripting formally the user behavior is tedious, DbC might be an alternative where you specify as much as you can, and then just re-run through the session. This implies, of course, a human tester.
>

That doesn't sound like the function's contract. That sounds like a
test case - of which you would have multiple, using different
"scripted session" inputs and different outputs (some of success, some
of expected failure).

Well, yes, you can view it as a testing technique; it assumes that scripting a session is often difficult for GUIs and sometimes harder (since combinatorial) than the implementation itself. What I saw people do is write the contracts, put the program in debug mode and let the human tester test it. Think of it as a random test where only checks are your pre/postconditions. The human annotator mimics a meaningful random walk and uses the application as s/he sees fit. I'm not championing this approach, just noting it here as a potential use case.

There's certainly benefits for the "contracts as additional tests"
viewpoint. But whenever that's proposed as what people understand by
DbC, the response is "no, it's not like that at all". So going back to
the "why isn't DbC more popular" question - because no-one can get a
clear handle on whether they are "like tests" or "like assertions" or
"something else" :-)
 
I think that it's a tool that you can use for many things:
* verifiable documentation
* deeper testing (every test case tests also other parts of the system, akin to asserts)
* automatic test generation
* hand-break akin to asserts

I find the first one, verifiable documentation, to be the most useful one in working with my team at the moment.

Cheers,
Marko

On Wed, 26 Sep 2018 at 10:59, Paul Moore <p.f.moore@gmail.com> wrote:
On Wed, 26 Sep 2018 at 09:45, Chris Angelico <rosuav@gmail.com> wrote:
>
> On Wed, Sep 26, 2018 at 6:36 PM Paul Moore <p.f.moore@gmail.com> wrote:
> >
> > On Wed, 26 Sep 2018 at 06:41, Chris Angelico <rosuav@gmail.com> wrote:
> > >
> > > On Wed, Sep 26, 2018 at 2:47 PM Marko Ristin-Kaufmann
> > > <marko.ristin@gmail.com> wrote:
> > > > * The contracts written in documentation as human text inevitably rot and they are much harder to maintain than automatically verified formal contracts.
> > >
> > > Agreed.
> >
> > Agreed, if contracts are automatically verified. But when runtime cost
> > comes up, people suggest that contracts can be disabled in production
> > code - which invalidates the "automatically verified" premise.
>
> Even if they're only verified as a dedicated testing pass ("okay,
> let's run the unit tests, let's run the contract verifier, let's run
> the type checker, cool, we're good"), they're still better off than
> unchecked comments/documentation in terms of code rot.

Absolutely. But if the contracts are checked at runtime, they are
precisely as good as tests - they will flag violations *in any
circumstances you check*. That's great, but nothing new. I understood
that one of the benefits of contracts was that it would handle cases
that you *forgot* to test - like assertions do, in essence - and would
need to be active in production (again, like assertions, if we assume
we've not explicitly chosen to run with assertions disabled) to get
that benefit.

There's certainly benefits for the "contracts as additional tests"
viewpoint. But whenever that's proposed as what people understand by
DbC, the response is "no, it's not like that at all". So going back to
the "why isn't DbC more popular" question - because no-one can get a
clear handle on whether they are "like tests" or "like assertions" or
"something else" :-)

Paul
_______________________________________________
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/