On Sun, Sep 30, 2018 at 12:17:25PM +1000, Chris Angelico wrote:

On Sun, Sep 30, 2018 at 11:54 AM Steven D'Aprano wrote:

...

This discussion is for those of us who would like to include DbC in our projects but don't like existing solutions. C++ being designed with a shovel is not relevant.

(Except in the sense that we should always be careful about piling on feature upon feature into Python.)

And as such, I do not want to see dedicated syntax for no purpose other than contracts.

That's a reasonable objection, except that contracts are a BIG purpose. Would you object to dedicated syntax for object oriented programming? (Classes and methods.) I hope not. Imagine if OOP in Python was limited to an API like this: MyClass = type(name="MyClass", parent=object) MyClass.add_method(__init__=lambda self, arg: setattr(self, "arg", arg)) MyClass.add_method(__str__=lambda self: "MyClass(%r)" % (self.arg,)) MyClass.add_method(spam=lambda self, x, y: (self.arg + x)/y) MyClass.add_method(eggs=lambda self, x, y: self.arg*x - y) MyClass.add_member(cheese='Cheddar') MyClass.add_member(aardvark=None) That's the situation we're in right now for contracts. It sucks and blows at the same time. Syntax matters, and sometimes without the right syntax, certain techniques and methodologies aren't practical. I know that adding syntax is a big step, and should be considered a last resort for when a library or even a built-in won't work. But adding contracts isn't a small benefit. Its not a magic bullet, nobody says that, but I would say that contracts as a feature is *much* bigger and more important than (say) docstrings, and we have dedicated syntax for docstrings.

What I'm interested in is (a) whether something can and should be added to the stdlib, and (b) whether some specific (and probably small) aspect of it could benefit from language support. As a parallel example, consider type hints. The language has ZERO support for special syntax for a language of types.

That's a weird use of the word "ZERO" :-) def spam(x: int) -> float: y: int I count three special syntax forms for a language of types: - parameter type hints; - return type hints; - variable type hints. (Yes, I'm aware that *technically* we can put anything we like in the hints, they don't have to be used as type hints, but Guido has made it clear that such uses are definitely of second-rate importance and only grudgingly supported.)

What you have is simple, straight-forward names like "List", and the normal behaviours that we already can do such as subscripting. There is language support, however, for attaching expressions to functions and their arguments.

Your point is taken that there is no separate syntax for referring to the types *themselves*, but then there's no need for such. int is int, whether you refer to the class "int" or the static type "int".

At the moment, I'm seeing decorator-based contracts as a clunky version of unit tests.

Contracts are not unit tests. Contracts and unit tests are complementary, and overlap somewhat, but they are the same. Unit tests only test the canned values you write in you tests. Contracts test real data you pass to your application. -- Steve

On Sun, Sep 30, 2018 at 02:50:28PM +1000, Chris Angelico wrote:

And yet all the examples I've seen have just been poor substitutes for unit tests. Can we get some examples that actually do a better job of selling contracts?

In no particular order... (1) Distance Unit tests are far away from the code you are looking at. Things which go together ought to be together, but typically unit tests are not just seperate from the thing they are testing, but in a completely different file. Contracts are right there, next to the thing they belong with, but without being mixed into the implementation of the method or function. (2) Self-documenting code Contracts are self-documenting code. Unit tests are not. Unit tests are full of boilerplate, creating instances, setting up test data, checking the result. Contracts simply cut to the chase and state the requirements and the promises made: the input must be a non-empty list the result will be a string starting with "Aardvark" as executable code. (3) The "Have you performed the *right* tests?" problem Unit tests are great, but they have a serious problem: they test ONLY the canned data you put in your test. For non-trivial functions, unit tests tell you nothing about the general behaviour of your function, only the specific behaviour with the given input: The key problem with testing is that a test (of any kind) that uses one particular set of inputs tells you nothing at all about the behaviour of the system or component when it is given a different set of inputs. http://thinkrelevance.com/blog/2013/11/26/better-than-unit-tests Contracts are one (partial) solution to this problem. If you have a unit test that does this: def test_spam(self): self.AssertEqual(spam(2).count("eggs"), 2) then it tests ONLY that spam(2) contains "eggs" twice, and that's it. It tells you nothing about whether spam(3) or spam(1028374527601) is correct. In fact, under Test Driven Development, it would be normal to write the test first, and then implement spam as a stub that does this: def spam(n): return "eggs eggs" proving my point that a passing test with one input doesn't mean the function is correct for another input. In contrast, the post-condition: def spam(n): ensure: result.count("eggs") == max(0, n) # implementation is tested on every invocation of spam (up to the point that you decide to disable post-condition checks). There is no need for separate tests for spam(2) and spam(3) and spam(1028374527601) unless you have some specific need for them. (Say, a regression test after fixing a particular bug.) (4) Inheritance Contracts are inherited, unit tests are not. (5) Unit tests and contracts are complementary, not alternatives Unit tests and contracts do overlap, and in the areas of overlap contracts are generally superior. But sometimes it is too hard to specify a contract in sufficient detail, and so unit tests are more appropriate. And for some especially simple functions, you can't specify the post-condition except by duplicating the implementation: def max(a, b): """Return the maximum of a and b.""" ensure: result == a if a >= b else b implementation: return a if a >= b else b In that case, a post-condition is a waste of time, and one should just unit test it. https://sebnozzi.github.io/362/contracts-replace-unit-tests/ Another way to think about it is that unit tests and contracts have different purposes. Pre-conditions and class invariants are a form of defensive programming that ensures that your prerequisites are met, that code is called with the correct parameters, etc. Unit tests are a way of doing spot tests that the code works with certain specified inputs. (6) Separation of concerns: function algorithm versus error checking Functions ought to validate their input, but doing so obfuscates the function implementation. Making that input validation a pre-condition separates the error checking and input validation from the algorithm proper, which helps make the code self-documenting. (7) You can't unit test loop invariants Some languages have support for testing loop invariants. You can't unit test loop invariants at all. https://en.wikipedia.org/wiki/Loop_invariant#Programming_language_support (8) Executable documentation Contracts are executable code that document what input is valid and what result is returned. Executable code is preferrable to dead comments because comments rot: "At Resolver we've found it useful to short-circuit any doubt and just refer to comments in code as 'lies'. " -- Michael Foord paraphrases Christian Muirhead on python-dev, 2009-03-22 Contracts can also be extracted by static tools. -- Steve

On Sun, Sep 30, 2018 at 06:33:08PM +1000, Chris Angelico wrote:

On Sun, Sep 30, 2018 at 6:03 PM Steven D'Aprano wrote:

...

On Sun, Sep 30, 2018 at 02:50:28PM +1000, Chris Angelico wrote:

...

And yet all the examples I've seen have just been poor substitutes for unit tests. Can we get some examples that actually do a better job of selling contracts?

In no particular order...

(1) Distance

Don't doctests deal with this too? With the exact same downsides?

I'm not sure that I understand the question. Yes, doctests are usually (but not always) close to the function too. That's a plus in favour of doctests. The downside is that if you want to show a metric tonne of examples, your docstring becomes huge. (Or move it out into a separate text file, which doctest can still run.) Sometimes its a real struggle to write a good docstring that isn't huge. But on the other hand, a good editor should allow you to collapse docstrings to a single line.

...

(2) Self-documenting code

Ditto

Sorry, are you asking if doctests are self-documenting code? Since they aren't part of the implementation of the function, no they aren't.

...

(3) The "Have you performed the *right* tests?" problem

Great if your contracts can actually be perfectly defined.

Why do you think the contracts have to be perfectly defined? I keep reading people accusing pro-Contracts people of being zealots who believe that Contracts are a magic bullet that solve every problem 100% perfectly. I don't know where people get this idea. Contracts are a tool, like unit tests. Just as a single unit test is better than no unit tests, and two unit tests is better than one, same goes for contracts. If you have an imperfect (incomplete, not buggy!) contract, that's better than *no* contract at all. If you would like to check five things, but can only specify three of them in a contract, that's better than not specifying any of them.

Every time a weird case is mentioned, those advocating contracts (mainly Marko) give examples showing "hey, contracts can do that too", and they're just testing specifics.

Can you give an example? Of course, contracts *can* be specific (at least in Eiffel), the syntax allows it. In pseudo-code: def spam(x, y): require: if x is None: y > 0 # implementation goes here... The precondition checks that if x is None, y must be greater than zero. On its own, that's not a very good contract since it tells us nothing about the case when x is not None, but (as per my comments above) its better than nothing. How would you do this as a unit test? For starters, you need to move the precondition into the function implementation block, giving it an explicit raise: def spam(x, y): if x is None and y <= 0: raise SomeError # implementation continues... Now you've given up any hope of disabling that specific check, short of editing the source code. Then you have to make a unit test to check that it works the way you think it does: def test_if_x_is_none_y_cannot_be_negative_or_zero(self): # Naming things is hard... self.AssertRaise(SomeError, spam, None, -1) self.AssertRaise(SomeError, spam, None, 0) Great! Now we know that if spam is passed None and -1, or None and 0, it will correctly fail. But the unit test doesn't give us any confidence that None and -9137 will fail, since we haven't tested that case. Work-around: I often write unit tests like this: def test_if_x_is_none_y_cannot_be_negative_or_zero(self): for i in range(20): # I never know how many or how few values to check... n = random.randint(1, 1000) # or which values... self.AssertRaise(SomeError, spam, None, -n) self.AssertRaise(SomeError, spam, None, 0) As David points out in another post, it is true that Contracts also can only check the values that you actually pass to the application during the testing phase, but that is likely to be a larger set of values than those we put in the unit tests, which lets us gain confidence in the code more quickly and with less effort. Rather than write lots of explicit tests, we can just exercise the routine with realistic data and see if the contract checks fail. [...]

...

(6) Separation of concerns: function algorithm versus error checking

Agreed, so long as you can define the contract in a way that isn't just duplicating the function's own body.

Sure. That's a case where contracts aren't really suitable and a few unit tests would be appropriate, but that's only likely to apply to post-conditions, not pre-conditions.

Contracts are great for some situations, but I'm seeing a lot of cases where they're just plain not, yet advocates still say "use contracts, use contracts". Why?

For the same reason advocates say "Use tests, use tests" despite there being some cases where formal automated tests are too hard to use. Use contracts for the 90% of the cases they help, don't dismiss them because of the 10% of cases they don't. But honestly, if you prefer unit testing, that's great too. Nobody is trying to say that unit tests suck or that you shouldn't use them. This is about giving people the choice between contracts or unit tests or both. When I say "Use contracts", what I mean is "Use as many or as few contracts as makes sense for your application, according to your taste". Right now, that choice is very restrictive, and I personally don't like the look and feel of existing contract libraries. I would give my right eye for Cobra-like syntax for contracts in Python: http://cobra-language.com/trac/cobra/wiki/Contracts -- Steve

Hi, I implement my own design-by-contract module. You can see it here: https://github.com/AlanCristhian/eiffel I did this as an experiment, I have no real life experience with the design by contract approach. El mar., 2 oct. 2018 a las 2:28, Marko Ristin-Kaufmann (< marko.ristin@gmail.com>) escribió:

Hi Steven (D'Aprano),

Right now, that choice is very restrictive, and I personally don't like

...

the look and feel of existing contract libraries. I would give my right eye for Cobra-like syntax for contracts in Python:

http://cobra-language.com/trac/cobra/wiki/Contracts

You might be interested in the discussion on transpiling contracts back and forth. See the other thread "Transpile contracts" and the issue in github: https://github.com/Parquery/icontract/issues/48 _______________________________________________ Python-ideas mailing list Python-ideas@python.org https://mail.python.org/mailman/listinfo/python-ideas Code of Conduct: http://python.org/psf/codeofconduct/

On Sun, Sep 30, 2018, 15:12 Stephen J. Turnbull < turnbull.stephen.fw@u.tsukuba.ac.jp> wrote:

Steven D'Aprano writes:

...

(4) Inheritance

Contracts are inherited, unit tests are not.

What does "inherited" mean? Just that methods that are not overridden retain their contracts?

Contracts are attached to interfaces, not to specifications. So when you have abstract base class, it defines contracts, and implementing classes must adhere to these contracts - the can only strengthen it, not weaken it. This way the user code need pnly be aware of the specification, not the implementation. So method that _are_ overridden retain their contracts. This is precisely like with types, since types are contracts (and vice versa, in a way). Elazar

Hi Steve (Turnbull), It sounds to me like the proof of the pudding for Python will be

annotating the ABC module with contracts. If that turns out to be a mostly fruitless exercise, it's hard to see how "real" contracts (vs. using asserts to "program in contracting style") are an important feature for Python in general.

Do you mean annotating an abstract class with contracts or annotating the actual functions in abc module? If you meant abstract classes in general, we do that for quite some time in our code base and had no problems so far. icontract already supports inheritance of contracts (as well as weakening/strengthening of the contracts). Inheriting of contracts also proved useful when you define an interface as a group of functions together with the contracts. You define the functions as an ABC with purely static methods (no state) and specify the contracts in that abstract class. The concrete classes implement the static functions, but don't need to repeat the contracts since they are inherited from the abstract class. I can copy/paste from our code base if you are interested. Cheers, Marko On Mon, 1 Oct 2018 at 15:09, Stephen J. Turnbull < turnbull.stephen.fw@u.tsukuba.ac.jp> wrote:

Elazar writes:

...

On Sun, Sep 30, 2018, 15:12 Stephen J. Turnbull < turnbull.stephen.fw@u.tsukuba.ac.jp> wrote:

...

...

What does "inherited" mean? Just that methods that are not overridden retain their contracts?

Contracts are attached to interfaces, not to specifications. So when you have abstract base class, it defines contracts, and implementing classes must adhere to these contracts - the can only strengthen it, not weaken it.

Stated in words, it sounds very reasonable.

Thinking about the implications for writing Python code, it sounds very constraining in the context of duck-typing. Or, to give a contrapositive example, you can write few, if any, contracts for dunders beyond their signatures. For example, len(x) + len(y) == len(x + y) is not an invariant over the classes that define __len__ (eg, set and dict, taking the "x + y" loosely for them). Of course if you restrict to Sequence, you can impose that invariant. The question is the balance.

It sounds to me like the proof of the pudding for Python will be annotating the ABC module with contracts. If that turns out to be a mostly fruitless exercise, it's hard to see how "real" contracts (vs. using asserts to "program in contracting style") are an important feature for Python in general.

That said, given the passion of the proponents, if we can come up with an attractive style for implementing contracts without a language change, I'd be +0 at least for adding a module to the stdlib. But this is a Python 3.9 project, given the state of the art.

...

This is precisely like with types, since types are contracts (and vice versa, in a way).

Not Python types (== classes), though. As a constraint on behavior, subclassing is purely nominal, although Python deliberately makes it tedious to turn a subclass of str into a clone of int.

Steve _______________________________________________ Python-ideas mailing list Python-ideas@python.org https://mail.python.org/mailman/listinfo/python-ideas Code of Conduct: http://python.org/psf/codeofconduct/

On 9/30/2018 8:11 AM, Stephen J. Turnbull wrote:

Steven D'Aprano writes:

...

(7) You can't unit test loop invariants

I don't see how a loop invariant can be elegantly specified without mixing it in to the implementation. Can you show an example of code written in a language with support for loop invariants *not* mixed into the implementation?

I'd be very interested in this, too. Any pointers? Eric

Steven D'Aprano writes:

I'm not sure how Stephen went from my comment that you can't unit test loop invariants to the idea that loop invariants aren't in the implementation.

Obviously I didn't. I went from your statement that (1) contracts are separated from the implementation and the inference from (7) that contracts can specify loop invariants to the question of whether both can be true at the same time.

Obviously the loop invariant has to be attached to the loop. You can't attach a loop invariant to a method or class, because they might contain more than one loop.

That's not so obvious. At least in theory you could provide a label for the loop (Common Lisp for example has named blocks), and refer to that in an "invariant" section outside the function. I couldn't imagine a pleasing syntax without help from someone who's used contracts, though. That's why I asked. It's not a big deal that they aren't both simultaneously true in practice, it's just that there are a huge number of abstract claims being made for contracts, and for my money they don't look anywhere near as much of an improvement over asserts in practice as they sound in the abstract, in decorator form or in Eiffel syntax. As I said before, since a number of people are asking for them, I'm at least +0 on including a contracts module with attractive syntax for contracts in the stdlib on the general principle that we should provide batteries for as many common patterns or styles as possible. But I'm not ready to go that high for language changes for this purpose, at least not yet. Since I don't expect to be using contracts any time soon, I'll leave the definition of "attractive" up to those who expect to have to read them frequently. Steve

On 30Sep2018 12:17, Chris Angelico wrote:

At the moment, I'm seeing decorator-based contracts as a clunky version of unit tests. We already have "inline unit testing" - it's called doctest - and I haven't seen anything pinned down as "hey, this is what it'd take to make contracts more viable". Certainly nothing that couldn't be done as a third-party package. But I'm still open to being swayed on that point.

Decorator based contracts are very little like clunky unit tests to me. I'm basing my opinion on the icontracts pip package, which I'm going to start using. In case you've been looking at something different, it provides a small number of decorators including @pre(test-function) and @post(test-function) and the class invariant decorator @inv, with good error messages for violations. They are _functionally_ like putting assertions in your code at the start and end of your functions, but have some advantages: - they're exposed, not buried inside the function, where they're easy to see and can be considered as contracts - they run on _every_ function call, not just during your testing, and get turned off just like assertions do: when you run Python with the -O (optimise) option. (There's some more tuning available too.) - the assertions make qualitative statements about the object/parameter state in the form "the state is consistent if these things apply"; tests tend to say "here's a situation, do these things and examine these results". You need to invent the situations and the results, rather than making general statements about the purpose and functional semantics of the class. They're different to both unit tests _and_ doctests because they get exercised during normal code execution. Both unit tests and doctests run _only_ during your test phase, with only whatever test scenarios you have devised. The difficulty with unit tests and doctests (both of which I use) and also integration tests is making something small enough to run but big/wide enough to cover all the relevant cases. They _do not_ run against all your real world data. It can be quite hard to apply them to your real world data. Also, all the @pre/@post/@inv stuff will run _during_ your unit tests and doctests as well, so they get included in your test regime for free. I've got a few classes which have a selftest method whose purpose is to confirm correctness of the instance state, and I call that from a few methods at start and end, particularly those for which unit tests have been hard to write or I know are inadequately covered (and probably never will be because devising a sufficient test case is impractical, especially for hard to envisage corner cases). The icontracts module will be very helpful to me here: I can pull out the self-test function as the class invariant, and make a bunch of @pre/@post assertions corresponding the the method semantic definition. The flip side of this is that there's no case for language changes in what I say above: the decorators look pretty good to my eye. Cheers, Cameron Simpson

On 9/30/2018 2:19 AM, Marko Ristin-Kaufmann wrote:

The library name will also need to change. When I started developing it, I was not aware of Java icontract library. It will be probably renamed to "pcontract" or any other suggested better name :)

'icontract' immediatly looks to me like 'internet contract', which is wrong (the Apple effect). I am guessing 'interface contract'. I would prefer 'pycontract'. -- Terry Jan Reedy