Re: [Python-ideas] Simplicity of C (was why is design-by-contracts not widely)
Oh FFS. You couldn't make the effort to read the very next sentence, let alone the next paragraph, before responding? -- cheers, Hugh Fisher
On Sat, Sep 29, 2018 at 09:50:27AM +1000, Hugh Fisher wrote:
Oh FFS. You couldn't make the effort to read the very next sentence, let alone the next paragraph, before responding?
Hugh, your mind-reading powers are fading. I read not just the next sentence and paragraph but the entire post. They aren't relevant to my comment: I still disagreed with your description of C (in particular) as "simple" although I'm willing to accept that Python is *relatively* simple in some ways. Just because I disagree with you doesn't mean I didn't read your post. The next sentence was: [Hugh] When starting a programming project in C or Python, there's maybe a brief discussion about C99 or C11, or Python 3.5 or 3.6, but that's it. There's one way to do it. followed by criticism of C++ for being "designed with a shovel rather than a chisel", and the comment: [Hugh] C++ programming projects often start by specifying exactly which bits of the language the programming team will be allowed to use. And in-house style guides for Python often do the same. For example, Google's style-guide for Python bans the use of "Power features" such as custom metaclasses, access to bytecode, on-the-fly compilation, dynamic inheritance, object reparenting, import hacks, reflection, modification of system internals, etc. https://github.com/google/styleguide/blob/gh-pages/pyguide.md#219-power-feat... Other choices include whether to use a functional style or object- oriented style or both. Design By Contract is a methodology. People already decide whether to use TDD or design up front (or don't decide on any methodology at all and wing it). They can already decide on using Design By Contract, if they like the existing solutions for it. This discussion is for those of us who would like to include DbC in our projects but don't like existing solutions. C++ being designed with a shovel is not relevant. (Except in the sense that we should always be careful about piling on feature upon feature into Python.) -- Steve
On Sun, Sep 30, 2018 at 11:54 AM Steven D'Aprano <steve@pearwood.info> wrote:
And as such, I do not want to see dedicated syntax for no purpose other than contracts. What I'm interested in is (a) whether something can and should be added to the stdlib, and (b) whether some specific (and probably small) aspect of it could benefit from language support. As a parallel example, consider type hints. The language has ZERO support for special syntax for a language of types. What you have is simple, straight-forward names like "List", and the normal behaviours that we already can do such as subscripting. There is language support, however, for attaching expressions to functions and their arguments. At the moment, I'm seeing decorator-based contracts as a clunky version of unit tests. We already have "inline unit testing" - it's called doctest - and I haven't seen anything pinned down as "hey, this is what it'd take to make contracts more viable". Certainly nothing that couldn't be done as a third-party package. But I'm still open to being swayed on that point. ChrisA
On Sun, Sep 30, 2018 at 12:17:25PM +1000, Chris Angelico wrote:
That's a reasonable objection, except that contracts are a BIG purpose. Would you object to dedicated syntax for object oriented programming? (Classes and methods.) I hope not. Imagine if OOP in Python was limited to an API like this: MyClass = type(name="MyClass", parent=object) MyClass.add_method(__init__=lambda self, arg: setattr(self, "arg", arg)) MyClass.add_method(__str__=lambda self: "MyClass(%r)" % (self.arg,)) MyClass.add_method(spam=lambda self, x, y: (self.arg + x)/y) MyClass.add_method(eggs=lambda self, x, y: self.arg*x - y) MyClass.add_member(cheese='Cheddar') MyClass.add_member(aardvark=None) That's the situation we're in right now for contracts. It sucks and blows at the same time. Syntax matters, and sometimes without the right syntax, certain techniques and methodologies aren't practical. I know that adding syntax is a big step, and should be considered a last resort for when a library or even a built-in won't work. But adding contracts isn't a small benefit. Its not a magic bullet, nobody says that, but I would say that contracts as a feature is *much* bigger and more important than (say) docstrings, and we have dedicated syntax for docstrings.
That's a weird use of the word "ZERO" :-) def spam(x: int) -> float: y: int I count three special syntax forms for a language of types: - parameter type hints; - return type hints; - variable type hints. (Yes, I'm aware that *technically* we can put anything we like in the hints, they don't have to be used as type hints, but Guido has made it clear that such uses are definitely of second-rate importance and only grudgingly supported.)
Your point is taken that there is no separate syntax for referring to the types *themselves*, but then there's no need for such. int is int, whether you refer to the class "int" or the static type "int".
At the moment, I'm seeing decorator-based contracts as a clunky version of unit tests.
Contracts are not unit tests. Contracts and unit tests are complementary, and overlap somewhat, but they are the same. Unit tests only test the canned values you write in you tests. Contracts test real data you pass to your application. -- Steve
On Sun, Sep 30, 2018 at 2:27 PM Steven D'Aprano <steve@pearwood.info> wrote:
I think that's a matter of debate, but sure, let's accept that contracts are big.
TBH I don't think contracts are nearly as big a tool as classes are.
Actually, that's not far from the way JavaScript's class hierarchy was, up until ES2015. Which suggests that a class keyword is a convenience, but not actually essential.
Hmm, what we really have is just string literals, plus a bit of magic that says "if the first thing in a function/class is a string literal, we save it". But sure.
That's exactly my point though. The syntax is for "attach this thing to that function". There is no syntax for a language of types, a type algebra syntax. We don't have a syntax that says "tuple containing int, two strings, and float". The syntactic support is the very smallest it can be - and, as you say, it's not actually restricted to type hints at all. If I'm reading the dates correctly, annotations were completely non-specific from 2006 (PEP 3107) until 2014 (PEP 484). So what is the smallest piece of syntactic support that would enable decent DbC? And "none at all" is a valid response, although I think in this case it's incorrect.
The static type "int" is one of the simplest possible type declarations. There are much more complicated options.
And yet all the examples I've seen have just been poor substitutes for unit tests. Can we get some examples that actually do a better job of selling contracts? ChrisA
On Sun, Sep 30, 2018 at 02:50:28PM +1000, Chris Angelico wrote:
In no particular order... (1) Distance Unit tests are far away from the code you are looking at. Things which go together ought to be together, but typically unit tests are not just seperate from the thing they are testing, but in a completely different file. Contracts are right there, next to the thing they belong with, but without being mixed into the implementation of the method or function. (2) Self-documenting code Contracts are self-documenting code. Unit tests are not. Unit tests are full of boilerplate, creating instances, setting up test data, checking the result. Contracts simply cut to the chase and state the requirements and the promises made: the input must be a non-empty list the result will be a string starting with "Aardvark" as executable code. (3) The "Have you performed the *right* tests?" problem Unit tests are great, but they have a serious problem: they test ONLY the canned data you put in your test. For non-trivial functions, unit tests tell you nothing about the general behaviour of your function, only the specific behaviour with the given input: The key problem with testing is that a test (of any kind) that uses one particular set of inputs tells you nothing at all about the behaviour of the system or component when it is given a different set of inputs. http://thinkrelevance.com/blog/2013/11/26/better-than-unit-tests Contracts are one (partial) solution to this problem. If you have a unit test that does this: def test_spam(self): self.AssertEqual(spam(2).count("eggs"), 2) then it tests ONLY that spam(2) contains "eggs" twice, and that's it. It tells you nothing about whether spam(3) or spam(1028374527601) is correct. In fact, under Test Driven Development, it would be normal to write the test first, and then implement spam as a stub that does this: def spam(n): return "eggs eggs" proving my point that a passing test with one input doesn't mean the function is correct for another input. In contrast, the post-condition: def spam(n): ensure: result.count("eggs") == max(0, n) # implementation is tested on every invocation of spam (up to the point that you decide to disable post-condition checks). There is no need for separate tests for spam(2) and spam(3) and spam(1028374527601) unless you have some specific need for them. (Say, a regression test after fixing a particular bug.) (4) Inheritance Contracts are inherited, unit tests are not. (5) Unit tests and contracts are complementary, not alternatives Unit tests and contracts do overlap, and in the areas of overlap contracts are generally superior. But sometimes it is too hard to specify a contract in sufficient detail, and so unit tests are more appropriate. And for some especially simple functions, you can't specify the post-condition except by duplicating the implementation: def max(a, b): """Return the maximum of a and b.""" ensure: result == a if a >= b else b implementation: return a if a >= b else b In that case, a post-condition is a waste of time, and one should just unit test it. https://sebnozzi.github.io/362/contracts-replace-unit-tests/ Another way to think about it is that unit tests and contracts have different purposes. Pre-conditions and class invariants are a form of defensive programming that ensures that your prerequisites are met, that code is called with the correct parameters, etc. Unit tests are a way of doing spot tests that the code works with certain specified inputs. (6) Separation of concerns: function algorithm versus error checking Functions ought to validate their input, but doing so obfuscates the function implementation. Making that input validation a pre-condition separates the error checking and input validation from the algorithm proper, which helps make the code self-documenting. (7) You can't unit test loop invariants Some languages have support for testing loop invariants. You can't unit test loop invariants at all. https://en.wikipedia.org/wiki/Loop_invariant#Programming_language_support (8) Executable documentation Contracts are executable code that document what input is valid and what result is returned. Executable code is preferrable to dead comments because comments rot: "At Resolver we've found it useful to short-circuit any doubt and just refer to comments in code as 'lies'. " -- Michael Foord paraphrases Christian Muirhead on python-dev, 2009-03-22 Contracts can also be extracted by static tools. -- Steve
On Sun, Sep 30, 2018 at 6:03 PM Steven D'Aprano <steve@pearwood.info> wrote:
Don't doctests deal with this too? With the exact same downsides?
(2) Self-documenting code
Ditto
(3) The "Have you performed the *right* tests?" problem
Great if your contracts can actually be perfectly defined. Every time a weird case is mentioned, those advocating contracts (mainly Marko) give examples showing "hey, contracts can do that too", and they're just testing specifics.
(4) Inheritance
Okay, that one I 100% grant you.
(5) Unit tests and contracts are complementary, not alternatives
That I agree with.
(6) Separation of concerns: function algorithm versus error checking
Agreed, so long as you can define the contract in a way that isn't just duplicating the function's own body.
(7) You can't unit test loop invariants
Sure.
(8) Executable documentation
Granted, but there are many forms of that. Contracts are great for some situations, but I'm seeing a lot of cases where they're just plain not, yet advocates still say "use contracts, use contracts". Why? ChrisA
On Sun, Sep 30, 2018 at 06:33:08PM +1000, Chris Angelico wrote:
I'm not sure that I understand the question. Yes, doctests are usually (but not always) close to the function too. That's a plus in favour of doctests. The downside is that if you want to show a metric tonne of examples, your docstring becomes huge. (Or move it out into a separate text file, which doctest can still run.) Sometimes its a real struggle to write a good docstring that isn't huge. But on the other hand, a good editor should allow you to collapse docstrings to a single line.
(2) Self-documenting code
Ditto
Sorry, are you asking if doctests are self-documenting code? Since they aren't part of the implementation of the function, no they aren't.
(3) The "Have you performed the *right* tests?" problem
Great if your contracts can actually be perfectly defined.
Why do you think the contracts have to be perfectly defined? I keep reading people accusing pro-Contracts people of being zealots who believe that Contracts are a magic bullet that solve every problem 100% perfectly. I don't know where people get this idea. Contracts are a tool, like unit tests. Just as a single unit test is better than no unit tests, and two unit tests is better than one, same goes for contracts. If you have an imperfect (incomplete, not buggy!) contract, that's better than *no* contract at all. If you would like to check five things, but can only specify three of them in a contract, that's better than not specifying any of them.
Can you give an example? Of course, contracts *can* be specific (at least in Eiffel), the syntax allows it. In pseudo-code: def spam(x, y): require: if x is None: y > 0 # implementation goes here... The precondition checks that if x is None, y must be greater than zero. On its own, that's not a very good contract since it tells us nothing about the case when x is not None, but (as per my comments above) its better than nothing. How would you do this as a unit test? For starters, you need to move the precondition into the function implementation block, giving it an explicit raise: def spam(x, y): if x is None and y <= 0: raise SomeError # implementation continues... Now you've given up any hope of disabling that specific check, short of editing the source code. Then you have to make a unit test to check that it works the way you think it does: def test_if_x_is_none_y_cannot_be_negative_or_zero(self): # Naming things is hard... self.AssertRaise(SomeError, spam, None, -1) self.AssertRaise(SomeError, spam, None, 0) Great! Now we know that if spam is passed None and -1, or None and 0, it will correctly fail. But the unit test doesn't give us any confidence that None and -9137 will fail, since we haven't tested that case. Work-around: I often write unit tests like this: def test_if_x_is_none_y_cannot_be_negative_or_zero(self): for i in range(20): # I never know how many or how few values to check... n = random.randint(1, 1000) # or which values... self.AssertRaise(SomeError, spam, None, -n) self.AssertRaise(SomeError, spam, None, 0) As David points out in another post, it is true that Contracts also can only check the values that you actually pass to the application during the testing phase, but that is likely to be a larger set of values than those we put in the unit tests, which lets us gain confidence in the code more quickly and with less effort. Rather than write lots of explicit tests, we can just exercise the routine with realistic data and see if the contract checks fail. [...]
Sure. That's a case where contracts aren't really suitable and a few unit tests would be appropriate, but that's only likely to apply to post-conditions, not pre-conditions.
For the same reason advocates say "Use tests, use tests" despite there being some cases where formal automated tests are too hard to use. Use contracts for the 90% of the cases they help, don't dismiss them because of the 10% of cases they don't. But honestly, if you prefer unit testing, that's great too. Nobody is trying to say that unit tests suck or that you shouldn't use them. This is about giving people the choice between contracts or unit tests or both. When I say "Use contracts", what I mean is "Use as many or as few contracts as makes sense for your application, according to your taste". Right now, that choice is very restrictive, and I personally don't like the look and feel of existing contract libraries. I would give my right eye for Cobra-like syntax for contracts in Python: http://cobra-language.com/trac/cobra/wiki/Contracts -- Steve
Hi Steven (D'Aprano), Right now, that choice is very restrictive, and I personally don't like
You might be interested in the discussion on transpiling contracts back and forth. See the other thread "Transpile contracts" and the issue in github: https://github.com/Parquery/icontract/issues/48
Hi, I implement my own design-by-contract module. You can see it here: https://github.com/AlanCristhian/eiffel I did this as an experiment, I have no real life experience with the design by contract approach. El mar., 2 oct. 2018 a las 2:28, Marko Ristin-Kaufmann (< marko.ristin@gmail.com>) escribió:
Steven D'Aprano writes:
(1) Distance
(2) Self-documenting code
(3) The "Have you performed the *right* tests?" problem
(4) Inheritance
Contracts are inherited, unit tests are not.
What does "inherited" mean? Just that methods that are not overridden retain their contracts?
There's nothing in the Eiffel syntax that distinguishes *which* contract(s) was (were) violated. Is there some magic? Or does the Eiffel process just die? (ISTR that is a typical error "recovery" approach in systems implemented in Eiffel, maybe from the Beautiful Code book?)
(7) You can't unit test loop invariants
I don't see how a loop invariant can be elegantly specified without mixing it in to the implementation. Can you show an example of code written in a language with support for loop invariants *not* mixed into the implementation?
(8) Executable documentation
I don't see how any of your points fail to be satisfied by use of asserts with a convention that (except for loop invariants) they're placed either at the beginning or the end of the function, depending on whether they're pre- or post-conditions. This requires a single- exit style which may sometimes be unnatural, of course. AFAICS you can program in contract style already in Python. Contracts involving both beginning and end state would require annoying local variables, definitely. But other than that, all I see is a desire for unnecessary syntax, or stdlib, support.
On Sun, Sep 30, 2018, 15:12 Stephen J. Turnbull < turnbull.stephen.fw@u.tsukuba.ac.jp> wrote:
Contracts are attached to interfaces, not to specifications. So when you have abstract base class, it defines contracts, and implementing classes must adhere to these contracts - the can only strengthen it, not weaken it. This way the user code need pnly be aware of the specification, not the implementation. So method that _are_ overridden retain their contracts. This is precisely like with types, since types are contracts (and vice versa, in a way). Elazar
Elazar writes:
On Sun, Sep 30, 2018, 15:12 Stephen J. Turnbull < turnbull.stephen.fw@u.tsukuba.ac.jp> wrote:
Stated in words, it sounds very reasonable. Thinking about the implications for writing Python code, it sounds very constraining in the context of duck-typing. Or, to give a contrapositive example, you can write few, if any, contracts for dunders beyond their signatures. For example, len(x) + len(y) == len(x + y) is not an invariant over the classes that define __len__ (eg, set and dict, taking the "x + y" loosely for them). Of course if you restrict to Sequence, you can impose that invariant. The question is the balance. It sounds to me like the proof of the pudding for Python will be annotating the ABC module with contracts. If that turns out to be a mostly fruitless exercise, it's hard to see how "real" contracts (vs. using asserts to "program in contracting style") are an important feature for Python in general. That said, given the passion of the proponents, if we can come up with an attractive style for implementing contracts without a language change, I'd be +0 at least for adding a module to the stdlib. But this is a Python 3.9 project, given the state of the art.
This is precisely like with types, since types are contracts (and vice versa, in a way).
Not Python types (== classes), though. As a constraint on behavior, subclassing is purely nominal, although Python deliberately makes it tedious to turn a subclass of str into a clone of int. Steve
Hi Steve (Turnbull), It sounds to me like the proof of the pudding for Python will be
Do you mean annotating an abstract class with contracts or annotating the actual functions in abc module? If you meant abstract classes in general, we do that for quite some time in our code base and had no problems so far. icontract already supports inheritance of contracts (as well as weakening/strengthening of the contracts). Inheriting of contracts also proved useful when you define an interface as a group of functions together with the contracts. You define the functions as an ABC with purely static methods (no state) and specify the contracts in that abstract class. The concrete classes implement the static functions, but don't need to repeat the contracts since they are inherited from the abstract class. I can copy/paste from our code base if you are interested. Cheers, Marko On Mon, 1 Oct 2018 at 15:09, Stephen J. Turnbull < turnbull.stephen.fw@u.tsukuba.ac.jp> wrote:
Marko Ristin-Kaufmann writes:
Do you mean annotating an abstract class with contracts or annotating the actual functions in abc module?
I mean the functions in the abc module.
I can envision how that would work in a situation with application- specific classes, even in a deep hierarchy, with some discipline. However, Python does not necessarily conform to that kind of discipline. I suspect that Pythonic programming styles are not terribly amenable to inheritance of contracts. I think the ABC module is most likely to provide a strong evidence that I'm wrong: if contract inheritance, and in general the contracting style of programming, works well for that module, it will work for anything written in Python. By the way, I'm not asking you to do this; you've already provided both the icontract module and a rather complete annotation of a sizeable stdlib module. Your work is appreciated! Of course I'd be happy to see a similar treatment of abc, but it's also a reasonable candidate for thought experiments, or a sample of a few cases. Steve
On Sun, Sep 30, 2018 at 06:19:11PM -0400, Eric V. Smith wrote:
I'm not sure how Stephen went from my comment that you can't unit test loop invariants to the idea that loop invariants aren't in the implementation. Obviously the loop invariant has to be attached to the loop. You can't attach a loop invariant to a method or class, because they might contain more than one loop. https://en.wikipedia.org/wiki/Loop_invariant#Eiffel from x := 0 invariant x <= 10 until x > 10 loop x := x + 1 end -- Steve
Steven D'Aprano writes:
Obviously I didn't. I went from your statement that (1) contracts are separated from the implementation and the inference from (7) that contracts can specify loop invariants to the question of whether both can be true at the same time.
That's not so obvious. At least in theory you could provide a label for the loop (Common Lisp for example has named blocks), and refer to that in an "invariant" section outside the function. I couldn't imagine a pleasing syntax without help from someone who's used contracts, though. That's why I asked. It's not a big deal that they aren't both simultaneously true in practice, it's just that there are a huge number of abstract claims being made for contracts, and for my money they don't look anywhere near as much of an improvement over asserts in practice as they sound in the abstract, in decorator form or in Eiffel syntax. As I said before, since a number of people are asking for them, I'm at least +0 on including a contracts module with attractive syntax for contracts in the stdlib on the general principle that we should provide batteries for as many common patterns or styles as possible. But I'm not ready to go that high for language changes for this purpose, at least not yet. Since I don't expect to be using contracts any time soon, I'll leave the definition of "attractive" up to those who expect to have to read them frequently. Steve
On Tue, Oct 02, 2018 at 02:43:57AM +0900, Stephen J. Turnbull wrote:
Okay, fair enough. The loop invariant is separate in the sense that it is in a distinctly named block. But it connected in the sense that the loop invariant is attached to the loop, which is inside the function implementation, rather than in a separate block outside of the implementation. In Python terms, we might say something like: for x in sequence: BLOCK else: BLOCK invariant: BLOCK Static tools could parse the source and extract the invariant blocks, editors could allow you to collapse them out of sight. But unlike the (hypothetical) require and ensure blocks, its right there inside the function implementation. We presumably wouldn't want the invariant to be far away from the loop: ... code ... more code for x in sequence: BLOCK ... more code ... more code ... # and far, far away invariant: BLOCK Even if we had some mechanism to connect the two (some sort of label, as you suggest) it would suck from a readability perspective to have to jump backwards and forwards from the loop to the loop invariant. Things which are closely connected ought to be close together. -- Steve
On 30Sep2018 12:17, Chris Angelico <rosuav@gmail.com> wrote:
Decorator based contracts are very little like clunky unit tests to me. I'm basing my opinion on the icontracts pip package, which I'm going to start using. In case you've been looking at something different, it provides a small number of decorators including @pre(test-function) and @post(test-function) and the class invariant decorator @inv, with good error messages for violations. They are _functionally_ like putting assertions in your code at the start and end of your functions, but have some advantages: - they're exposed, not buried inside the function, where they're easy to see and can be considered as contracts - they run on _every_ function call, not just during your testing, and get turned off just like assertions do: when you run Python with the -O (optimise) option. (There's some more tuning available too.) - the assertions make qualitative statements about the object/parameter state in the form "the state is consistent if these things apply"; tests tend to say "here's a situation, do these things and examine these results". You need to invent the situations and the results, rather than making general statements about the purpose and functional semantics of the class. They're different to both unit tests _and_ doctests because they get exercised during normal code execution. Both unit tests and doctests run _only_ during your test phase, with only whatever test scenarios you have devised. The difficulty with unit tests and doctests (both of which I use) and also integration tests is making something small enough to run but big/wide enough to cover all the relevant cases. They _do not_ run against all your real world data. It can be quite hard to apply them to your real world data. Also, all the @pre/@post/@inv stuff will run _during_ your unit tests and doctests as well, so they get included in your test regime for free. I've got a few classes which have a selftest method whose purpose is to confirm correctness of the instance state, and I call that from a few methods at start and end, particularly those for which unit tests have been hard to write or I know are inadequately covered (and probably never will be because devising a sufficient test case is impractical, especially for hard to envisage corner cases). The icontracts module will be very helpful to me here: I can pull out the self-test function as the class invariant, and make a bunch of @pre/@post assertions corresponding the the method semantic definition. The flip side of this is that there's no case for language changes in what I say above: the decorators look pretty good to my eye. Cheers, Cameron Simpson <cs@cskk.id.au>
Hi Cameron, Just a word of caution: I made a mistake and badly designed interface to icontract. It will change in the near future from: @post(lambda arg1, arg2, result: arg1 < result < arg2) most probably to: @ensures(lambda P: P.arg1 < result < P.arg2) This avoids any name conflicts with "result" if it's in the arguments and also many conflicts with web frameworks which frequently use "post". We will also add snapshotting before the function execution: @snapshot(lambda P, var2: set(arg2)) @ensures(lambda O, P: P.arg1 < result and result in O.var2) so that postcondition can deal with state transitions. There are also some other approaches in discussion. The library name will also need to change. When I started developing it, I was not aware of Java icontract library. It will be probably renamed to "pcontract" or any other suggested better name :) Please see the github issues for more details and current discussions: https://github.com/Parquery/icontract/issues On Sun, 30 Sep 2018 at 06:44, Cameron Simpson <cs@cskk.id.au> wrote:
On 9/30/2018 2:19 AM, Marko Ristin-Kaufmann wrote:
'icontract' immediatly looks to me like 'internet contract', which is wrong (the Apple effect). I am guessing 'interface contract'. I would prefer 'pycontract'. -- Terry Jan Reedy
On Sat, Sep 29, 2018 at 09:50:27AM +1000, Hugh Fisher wrote:
Oh FFS. You couldn't make the effort to read the very next sentence, let alone the next paragraph, before responding?
Hugh, your mind-reading powers are fading. I read not just the next sentence and paragraph but the entire post. They aren't relevant to my comment: I still disagreed with your description of C (in particular) as "simple" although I'm willing to accept that Python is *relatively* simple in some ways. Just because I disagree with you doesn't mean I didn't read your post. The next sentence was: [Hugh] When starting a programming project in C or Python, there's maybe a brief discussion about C99 or C11, or Python 3.5 or 3.6, but that's it. There's one way to do it. followed by criticism of C++ for being "designed with a shovel rather than a chisel", and the comment: [Hugh] C++ programming projects often start by specifying exactly which bits of the language the programming team will be allowed to use. And in-house style guides for Python often do the same. For example, Google's style-guide for Python bans the use of "Power features" such as custom metaclasses, access to bytecode, on-the-fly compilation, dynamic inheritance, object reparenting, import hacks, reflection, modification of system internals, etc. https://github.com/google/styleguide/blob/gh-pages/pyguide.md#219-power-feat... Other choices include whether to use a functional style or object- oriented style or both. Design By Contract is a methodology. People already decide whether to use TDD or design up front (or don't decide on any methodology at all and wing it). They can already decide on using Design By Contract, if they like the existing solutions for it. This discussion is for those of us who would like to include DbC in our projects but don't like existing solutions. C++ being designed with a shovel is not relevant. (Except in the sense that we should always be careful about piling on feature upon feature into Python.) -- Steve
On Sun, Sep 30, 2018 at 11:54 AM Steven D'Aprano <steve@pearwood.info> wrote:
And as such, I do not want to see dedicated syntax for no purpose other than contracts. What I'm interested in is (a) whether something can and should be added to the stdlib, and (b) whether some specific (and probably small) aspect of it could benefit from language support. As a parallel example, consider type hints. The language has ZERO support for special syntax for a language of types. What you have is simple, straight-forward names like "List", and the normal behaviours that we already can do such as subscripting. There is language support, however, for attaching expressions to functions and their arguments. At the moment, I'm seeing decorator-based contracts as a clunky version of unit tests. We already have "inline unit testing" - it's called doctest - and I haven't seen anything pinned down as "hey, this is what it'd take to make contracts more viable". Certainly nothing that couldn't be done as a third-party package. But I'm still open to being swayed on that point. ChrisA
On Sun, Sep 30, 2018 at 12:17:25PM +1000, Chris Angelico wrote:
That's a reasonable objection, except that contracts are a BIG purpose. Would you object to dedicated syntax for object oriented programming? (Classes and methods.) I hope not. Imagine if OOP in Python was limited to an API like this: MyClass = type(name="MyClass", parent=object) MyClass.add_method(__init__=lambda self, arg: setattr(self, "arg", arg)) MyClass.add_method(__str__=lambda self: "MyClass(%r)" % (self.arg,)) MyClass.add_method(spam=lambda self, x, y: (self.arg + x)/y) MyClass.add_method(eggs=lambda self, x, y: self.arg*x - y) MyClass.add_member(cheese='Cheddar') MyClass.add_member(aardvark=None) That's the situation we're in right now for contracts. It sucks and blows at the same time. Syntax matters, and sometimes without the right syntax, certain techniques and methodologies aren't practical. I know that adding syntax is a big step, and should be considered a last resort for when a library or even a built-in won't work. But adding contracts isn't a small benefit. Its not a magic bullet, nobody says that, but I would say that contracts as a feature is *much* bigger and more important than (say) docstrings, and we have dedicated syntax for docstrings.
That's a weird use of the word "ZERO" :-) def spam(x: int) -> float: y: int I count three special syntax forms for a language of types: - parameter type hints; - return type hints; - variable type hints. (Yes, I'm aware that *technically* we can put anything we like in the hints, they don't have to be used as type hints, but Guido has made it clear that such uses are definitely of second-rate importance and only grudgingly supported.)
Your point is taken that there is no separate syntax for referring to the types *themselves*, but then there's no need for such. int is int, whether you refer to the class "int" or the static type "int".
At the moment, I'm seeing decorator-based contracts as a clunky version of unit tests.
Contracts are not unit tests. Contracts and unit tests are complementary, and overlap somewhat, but they are the same. Unit tests only test the canned values you write in you tests. Contracts test real data you pass to your application. -- Steve
On Sun, Sep 30, 2018 at 2:27 PM Steven D'Aprano <steve@pearwood.info> wrote:
I think that's a matter of debate, but sure, let's accept that contracts are big.
TBH I don't think contracts are nearly as big a tool as classes are.
Actually, that's not far from the way JavaScript's class hierarchy was, up until ES2015. Which suggests that a class keyword is a convenience, but not actually essential.
Hmm, what we really have is just string literals, plus a bit of magic that says "if the first thing in a function/class is a string literal, we save it". But sure.
That's exactly my point though. The syntax is for "attach this thing to that function". There is no syntax for a language of types, a type algebra syntax. We don't have a syntax that says "tuple containing int, two strings, and float". The syntactic support is the very smallest it can be - and, as you say, it's not actually restricted to type hints at all. If I'm reading the dates correctly, annotations were completely non-specific from 2006 (PEP 3107) until 2014 (PEP 484). So what is the smallest piece of syntactic support that would enable decent DbC? And "none at all" is a valid response, although I think in this case it's incorrect.
The static type "int" is one of the simplest possible type declarations. There are much more complicated options.
And yet all the examples I've seen have just been poor substitutes for unit tests. Can we get some examples that actually do a better job of selling contracts? ChrisA
On Sun, Sep 30, 2018 at 02:50:28PM +1000, Chris Angelico wrote:
In no particular order... (1) Distance Unit tests are far away from the code you are looking at. Things which go together ought to be together, but typically unit tests are not just seperate from the thing they are testing, but in a completely different file. Contracts are right there, next to the thing they belong with, but without being mixed into the implementation of the method or function. (2) Self-documenting code Contracts are self-documenting code. Unit tests are not. Unit tests are full of boilerplate, creating instances, setting up test data, checking the result. Contracts simply cut to the chase and state the requirements and the promises made: the input must be a non-empty list the result will be a string starting with "Aardvark" as executable code. (3) The "Have you performed the *right* tests?" problem Unit tests are great, but they have a serious problem: they test ONLY the canned data you put in your test. For non-trivial functions, unit tests tell you nothing about the general behaviour of your function, only the specific behaviour with the given input: The key problem with testing is that a test (of any kind) that uses one particular set of inputs tells you nothing at all about the behaviour of the system or component when it is given a different set of inputs. http://thinkrelevance.com/blog/2013/11/26/better-than-unit-tests Contracts are one (partial) solution to this problem. If you have a unit test that does this: def test_spam(self): self.AssertEqual(spam(2).count("eggs"), 2) then it tests ONLY that spam(2) contains "eggs" twice, and that's it. It tells you nothing about whether spam(3) or spam(1028374527601) is correct. In fact, under Test Driven Development, it would be normal to write the test first, and then implement spam as a stub that does this: def spam(n): return "eggs eggs" proving my point that a passing test with one input doesn't mean the function is correct for another input. In contrast, the post-condition: def spam(n): ensure: result.count("eggs") == max(0, n) # implementation is tested on every invocation of spam (up to the point that you decide to disable post-condition checks). There is no need for separate tests for spam(2) and spam(3) and spam(1028374527601) unless you have some specific need for them. (Say, a regression test after fixing a particular bug.) (4) Inheritance Contracts are inherited, unit tests are not. (5) Unit tests and contracts are complementary, not alternatives Unit tests and contracts do overlap, and in the areas of overlap contracts are generally superior. But sometimes it is too hard to specify a contract in sufficient detail, and so unit tests are more appropriate. And for some especially simple functions, you can't specify the post-condition except by duplicating the implementation: def max(a, b): """Return the maximum of a and b.""" ensure: result == a if a >= b else b implementation: return a if a >= b else b In that case, a post-condition is a waste of time, and one should just unit test it. https://sebnozzi.github.io/362/contracts-replace-unit-tests/ Another way to think about it is that unit tests and contracts have different purposes. Pre-conditions and class invariants are a form of defensive programming that ensures that your prerequisites are met, that code is called with the correct parameters, etc. Unit tests are a way of doing spot tests that the code works with certain specified inputs. (6) Separation of concerns: function algorithm versus error checking Functions ought to validate their input, but doing so obfuscates the function implementation. Making that input validation a pre-condition separates the error checking and input validation from the algorithm proper, which helps make the code self-documenting. (7) You can't unit test loop invariants Some languages have support for testing loop invariants. You can't unit test loop invariants at all. https://en.wikipedia.org/wiki/Loop_invariant#Programming_language_support (8) Executable documentation Contracts are executable code that document what input is valid and what result is returned. Executable code is preferrable to dead comments because comments rot: "At Resolver we've found it useful to short-circuit any doubt and just refer to comments in code as 'lies'. " -- Michael Foord paraphrases Christian Muirhead on python-dev, 2009-03-22 Contracts can also be extracted by static tools. -- Steve
On Sun, Sep 30, 2018 at 6:03 PM Steven D'Aprano <steve@pearwood.info> wrote:
Don't doctests deal with this too? With the exact same downsides?
(2) Self-documenting code
Ditto
(3) The "Have you performed the *right* tests?" problem
Great if your contracts can actually be perfectly defined. Every time a weird case is mentioned, those advocating contracts (mainly Marko) give examples showing "hey, contracts can do that too", and they're just testing specifics.
(4) Inheritance
Okay, that one I 100% grant you.
(5) Unit tests and contracts are complementary, not alternatives
That I agree with.
(6) Separation of concerns: function algorithm versus error checking
Agreed, so long as you can define the contract in a way that isn't just duplicating the function's own body.
(7) You can't unit test loop invariants
Sure.
(8) Executable documentation
Granted, but there are many forms of that. Contracts are great for some situations, but I'm seeing a lot of cases where they're just plain not, yet advocates still say "use contracts, use contracts". Why? ChrisA
On Sun, Sep 30, 2018 at 06:33:08PM +1000, Chris Angelico wrote:
I'm not sure that I understand the question. Yes, doctests are usually (but not always) close to the function too. That's a plus in favour of doctests. The downside is that if you want to show a metric tonne of examples, your docstring becomes huge. (Or move it out into a separate text file, which doctest can still run.) Sometimes its a real struggle to write a good docstring that isn't huge. But on the other hand, a good editor should allow you to collapse docstrings to a single line.
(2) Self-documenting code
Ditto
Sorry, are you asking if doctests are self-documenting code? Since they aren't part of the implementation of the function, no they aren't.
(3) The "Have you performed the *right* tests?" problem
Great if your contracts can actually be perfectly defined.
Why do you think the contracts have to be perfectly defined? I keep reading people accusing pro-Contracts people of being zealots who believe that Contracts are a magic bullet that solve every problem 100% perfectly. I don't know where people get this idea. Contracts are a tool, like unit tests. Just as a single unit test is better than no unit tests, and two unit tests is better than one, same goes for contracts. If you have an imperfect (incomplete, not buggy!) contract, that's better than *no* contract at all. If you would like to check five things, but can only specify three of them in a contract, that's better than not specifying any of them.
Can you give an example? Of course, contracts *can* be specific (at least in Eiffel), the syntax allows it. In pseudo-code: def spam(x, y): require: if x is None: y > 0 # implementation goes here... The precondition checks that if x is None, y must be greater than zero. On its own, that's not a very good contract since it tells us nothing about the case when x is not None, but (as per my comments above) its better than nothing. How would you do this as a unit test? For starters, you need to move the precondition into the function implementation block, giving it an explicit raise: def spam(x, y): if x is None and y <= 0: raise SomeError # implementation continues... Now you've given up any hope of disabling that specific check, short of editing the source code. Then you have to make a unit test to check that it works the way you think it does: def test_if_x_is_none_y_cannot_be_negative_or_zero(self): # Naming things is hard... self.AssertRaise(SomeError, spam, None, -1) self.AssertRaise(SomeError, spam, None, 0) Great! Now we know that if spam is passed None and -1, or None and 0, it will correctly fail. But the unit test doesn't give us any confidence that None and -9137 will fail, since we haven't tested that case. Work-around: I often write unit tests like this: def test_if_x_is_none_y_cannot_be_negative_or_zero(self): for i in range(20): # I never know how many or how few values to check... n = random.randint(1, 1000) # or which values... self.AssertRaise(SomeError, spam, None, -n) self.AssertRaise(SomeError, spam, None, 0) As David points out in another post, it is true that Contracts also can only check the values that you actually pass to the application during the testing phase, but that is likely to be a larger set of values than those we put in the unit tests, which lets us gain confidence in the code more quickly and with less effort. Rather than write lots of explicit tests, we can just exercise the routine with realistic data and see if the contract checks fail. [...]
Sure. That's a case where contracts aren't really suitable and a few unit tests would be appropriate, but that's only likely to apply to post-conditions, not pre-conditions.
For the same reason advocates say "Use tests, use tests" despite there being some cases where formal automated tests are too hard to use. Use contracts for the 90% of the cases they help, don't dismiss them because of the 10% of cases they don't. But honestly, if you prefer unit testing, that's great too. Nobody is trying to say that unit tests suck or that you shouldn't use them. This is about giving people the choice between contracts or unit tests or both. When I say "Use contracts", what I mean is "Use as many or as few contracts as makes sense for your application, according to your taste". Right now, that choice is very restrictive, and I personally don't like the look and feel of existing contract libraries. I would give my right eye for Cobra-like syntax for contracts in Python: http://cobra-language.com/trac/cobra/wiki/Contracts -- Steve
Hi Steven (D'Aprano), Right now, that choice is very restrictive, and I personally don't like
You might be interested in the discussion on transpiling contracts back and forth. See the other thread "Transpile contracts" and the issue in github: https://github.com/Parquery/icontract/issues/48
Hi, I implement my own design-by-contract module. You can see it here: https://github.com/AlanCristhian/eiffel I did this as an experiment, I have no real life experience with the design by contract approach. El mar., 2 oct. 2018 a las 2:28, Marko Ristin-Kaufmann (< marko.ristin@gmail.com>) escribió:
Steven D'Aprano writes:
(1) Distance
(2) Self-documenting code
(3) The "Have you performed the *right* tests?" problem
(4) Inheritance
Contracts are inherited, unit tests are not.
What does "inherited" mean? Just that methods that are not overridden retain their contracts?
There's nothing in the Eiffel syntax that distinguishes *which* contract(s) was (were) violated. Is there some magic? Or does the Eiffel process just die? (ISTR that is a typical error "recovery" approach in systems implemented in Eiffel, maybe from the Beautiful Code book?)
(7) You can't unit test loop invariants
I don't see how a loop invariant can be elegantly specified without mixing it in to the implementation. Can you show an example of code written in a language with support for loop invariants *not* mixed into the implementation?
(8) Executable documentation
I don't see how any of your points fail to be satisfied by use of asserts with a convention that (except for loop invariants) they're placed either at the beginning or the end of the function, depending on whether they're pre- or post-conditions. This requires a single- exit style which may sometimes be unnatural, of course. AFAICS you can program in contract style already in Python. Contracts involving both beginning and end state would require annoying local variables, definitely. But other than that, all I see is a desire for unnecessary syntax, or stdlib, support.
On Sun, Sep 30, 2018, 15:12 Stephen J. Turnbull < turnbull.stephen.fw@u.tsukuba.ac.jp> wrote:
Contracts are attached to interfaces, not to specifications. So when you have abstract base class, it defines contracts, and implementing classes must adhere to these contracts - the can only strengthen it, not weaken it. This way the user code need pnly be aware of the specification, not the implementation. So method that _are_ overridden retain their contracts. This is precisely like with types, since types are contracts (and vice versa, in a way). Elazar
Elazar writes:
On Sun, Sep 30, 2018, 15:12 Stephen J. Turnbull < turnbull.stephen.fw@u.tsukuba.ac.jp> wrote:
Stated in words, it sounds very reasonable. Thinking about the implications for writing Python code, it sounds very constraining in the context of duck-typing. Or, to give a contrapositive example, you can write few, if any, contracts for dunders beyond their signatures. For example, len(x) + len(y) == len(x + y) is not an invariant over the classes that define __len__ (eg, set and dict, taking the "x + y" loosely for them). Of course if you restrict to Sequence, you can impose that invariant. The question is the balance. It sounds to me like the proof of the pudding for Python will be annotating the ABC module with contracts. If that turns out to be a mostly fruitless exercise, it's hard to see how "real" contracts (vs. using asserts to "program in contracting style") are an important feature for Python in general. That said, given the passion of the proponents, if we can come up with an attractive style for implementing contracts without a language change, I'd be +0 at least for adding a module to the stdlib. But this is a Python 3.9 project, given the state of the art.
This is precisely like with types, since types are contracts (and vice versa, in a way).
Not Python types (== classes), though. As a constraint on behavior, subclassing is purely nominal, although Python deliberately makes it tedious to turn a subclass of str into a clone of int. Steve
Hi Steve (Turnbull), It sounds to me like the proof of the pudding for Python will be
Do you mean annotating an abstract class with contracts or annotating the actual functions in abc module? If you meant abstract classes in general, we do that for quite some time in our code base and had no problems so far. icontract already supports inheritance of contracts (as well as weakening/strengthening of the contracts). Inheriting of contracts also proved useful when you define an interface as a group of functions together with the contracts. You define the functions as an ABC with purely static methods (no state) and specify the contracts in that abstract class. The concrete classes implement the static functions, but don't need to repeat the contracts since they are inherited from the abstract class. I can copy/paste from our code base if you are interested. Cheers, Marko On Mon, 1 Oct 2018 at 15:09, Stephen J. Turnbull < turnbull.stephen.fw@u.tsukuba.ac.jp> wrote:
Marko Ristin-Kaufmann writes:
Do you mean annotating an abstract class with contracts or annotating the actual functions in abc module?
I mean the functions in the abc module.
I can envision how that would work in a situation with application- specific classes, even in a deep hierarchy, with some discipline. However, Python does not necessarily conform to that kind of discipline. I suspect that Pythonic programming styles are not terribly amenable to inheritance of contracts. I think the ABC module is most likely to provide a strong evidence that I'm wrong: if contract inheritance, and in general the contracting style of programming, works well for that module, it will work for anything written in Python. By the way, I'm not asking you to do this; you've already provided both the icontract module and a rather complete annotation of a sizeable stdlib module. Your work is appreciated! Of course I'd be happy to see a similar treatment of abc, but it's also a reasonable candidate for thought experiments, or a sample of a few cases. Steve
On Sun, Sep 30, 2018 at 06:19:11PM -0400, Eric V. Smith wrote:
I'm not sure how Stephen went from my comment that you can't unit test loop invariants to the idea that loop invariants aren't in the implementation. Obviously the loop invariant has to be attached to the loop. You can't attach a loop invariant to a method or class, because they might contain more than one loop. https://en.wikipedia.org/wiki/Loop_invariant#Eiffel from x := 0 invariant x <= 10 until x > 10 loop x := x + 1 end -- Steve
Steven D'Aprano writes:
Obviously I didn't. I went from your statement that (1) contracts are separated from the implementation and the inference from (7) that contracts can specify loop invariants to the question of whether both can be true at the same time.
That's not so obvious. At least in theory you could provide a label for the loop (Common Lisp for example has named blocks), and refer to that in an "invariant" section outside the function. I couldn't imagine a pleasing syntax without help from someone who's used contracts, though. That's why I asked. It's not a big deal that they aren't both simultaneously true in practice, it's just that there are a huge number of abstract claims being made for contracts, and for my money they don't look anywhere near as much of an improvement over asserts in practice as they sound in the abstract, in decorator form or in Eiffel syntax. As I said before, since a number of people are asking for them, I'm at least +0 on including a contracts module with attractive syntax for contracts in the stdlib on the general principle that we should provide batteries for as many common patterns or styles as possible. But I'm not ready to go that high for language changes for this purpose, at least not yet. Since I don't expect to be using contracts any time soon, I'll leave the definition of "attractive" up to those who expect to have to read them frequently. Steve
On Tue, Oct 02, 2018 at 02:43:57AM +0900, Stephen J. Turnbull wrote:
Okay, fair enough. The loop invariant is separate in the sense that it is in a distinctly named block. But it connected in the sense that the loop invariant is attached to the loop, which is inside the function implementation, rather than in a separate block outside of the implementation. In Python terms, we might say something like: for x in sequence: BLOCK else: BLOCK invariant: BLOCK Static tools could parse the source and extract the invariant blocks, editors could allow you to collapse them out of sight. But unlike the (hypothetical) require and ensure blocks, its right there inside the function implementation. We presumably wouldn't want the invariant to be far away from the loop: ... code ... more code for x in sequence: BLOCK ... more code ... more code ... # and far, far away invariant: BLOCK Even if we had some mechanism to connect the two (some sort of label, as you suggest) it would suck from a readability perspective to have to jump backwards and forwards from the loop to the loop invariant. Things which are closely connected ought to be close together. -- Steve
On 30Sep2018 12:17, Chris Angelico <rosuav@gmail.com> wrote:
Decorator based contracts are very little like clunky unit tests to me. I'm basing my opinion on the icontracts pip package, which I'm going to start using. In case you've been looking at something different, it provides a small number of decorators including @pre(test-function) and @post(test-function) and the class invariant decorator @inv, with good error messages for violations. They are _functionally_ like putting assertions in your code at the start and end of your functions, but have some advantages: - they're exposed, not buried inside the function, where they're easy to see and can be considered as contracts - they run on _every_ function call, not just during your testing, and get turned off just like assertions do: when you run Python with the -O (optimise) option. (There's some more tuning available too.) - the assertions make qualitative statements about the object/parameter state in the form "the state is consistent if these things apply"; tests tend to say "here's a situation, do these things and examine these results". You need to invent the situations and the results, rather than making general statements about the purpose and functional semantics of the class. They're different to both unit tests _and_ doctests because they get exercised during normal code execution. Both unit tests and doctests run _only_ during your test phase, with only whatever test scenarios you have devised. The difficulty with unit tests and doctests (both of which I use) and also integration tests is making something small enough to run but big/wide enough to cover all the relevant cases. They _do not_ run against all your real world data. It can be quite hard to apply them to your real world data. Also, all the @pre/@post/@inv stuff will run _during_ your unit tests and doctests as well, so they get included in your test regime for free. I've got a few classes which have a selftest method whose purpose is to confirm correctness of the instance state, and I call that from a few methods at start and end, particularly those for which unit tests have been hard to write or I know are inadequately covered (and probably never will be because devising a sufficient test case is impractical, especially for hard to envisage corner cases). The icontracts module will be very helpful to me here: I can pull out the self-test function as the class invariant, and make a bunch of @pre/@post assertions corresponding the the method semantic definition. The flip side of this is that there's no case for language changes in what I say above: the decorators look pretty good to my eye. Cheers, Cameron Simpson <cs@cskk.id.au>
Hi Cameron, Just a word of caution: I made a mistake and badly designed interface to icontract. It will change in the near future from: @post(lambda arg1, arg2, result: arg1 < result < arg2) most probably to: @ensures(lambda P: P.arg1 < result < P.arg2) This avoids any name conflicts with "result" if it's in the arguments and also many conflicts with web frameworks which frequently use "post". We will also add snapshotting before the function execution: @snapshot(lambda P, var2: set(arg2)) @ensures(lambda O, P: P.arg1 < result and result in O.var2) so that postcondition can deal with state transitions. There are also some other approaches in discussion. The library name will also need to change. When I started developing it, I was not aware of Java icontract library. It will be probably renamed to "pcontract" or any other suggested better name :) Please see the github issues for more details and current discussions: https://github.com/Parquery/icontract/issues On Sun, 30 Sep 2018 at 06:44, Cameron Simpson <cs@cskk.id.au> wrote:
On 9/30/2018 2:19 AM, Marko Ristin-Kaufmann wrote:
'icontract' immediatly looks to me like 'internet contract', which is wrong (the Apple effect). I am guessing 'interface contract'. I would prefer 'pycontract'. -- Terry Jan Reedy
participants (10)
-
Alan Cristhian -
Cameron Simpson -
Chris Angelico -
Elazar -
Eric V. Smith -
Hugh Fisher -
Marko Ristin-Kaufmann -
Stephen J. Turnbull -
Steven D'Aprano -
Terry Reedy