Re: [Python-ideas] Pre-conditions and post-conditions
I don't think that being useful by itself should be enough. I think new features should also be "Pythonic" and I don't see design by contract notation as a good fit. For example, C has the useful & operator which lets you pass &foo as a pointer/array argument despite foo being a scalar, so assignment to bar[0] in the called function actually sets the value of foo. It might be possible to create some kind of aliasing operator for Python so that two or more variables were bound to the same location, but would we want it? No, because Python is not intended for that style of programming. For another example, GPU shading languages have the special keywords uniform and varying for distinguishing definitions that won't change across parallel invocations and definitions that will. Demonstrably very useful in computer games and supercomputer number crunching, so why doesn't Python have those keywords? Because it's not designed to be used for such. For design by contract, as others have noted Python assert statements work fine for simple preconditions and postconditions. I don't see any significant difference in readability between existing def foo(x, y): assert(x > 0) # Do stuff assert(x == y) and new style def foo(x, y): require: x > 0 # Do stuff ensure: x == y Yes there's more to design by contract than simple assertions, but it's not just adding syntax. Meyer often uses the special "old" construct in his post condition examples, a trivial example being ensure count = old.count + 1 How do we do that in Python? And another part of design by contract (at least according to Meyer) is that it's not enough to just raise an exception, but there must be a guarantee that it is handled and the post conditions and/or invariants restored. So there's more syntax for "rescue" and "retry" If you want to do simple pre and post conditions, Python already has assert. If you want to go full design by contract, there's no law saying that Python is the only programming language allowed. Instead of trying to graft new and IMHO alien concepts onto Python, what's wrong with Eiffel? -- cheers, Hugh Fisher
On 08/29/2018 04:53 PM, Hugh Fisher wrote:
From: Marko Ristin-Kaufmann:
I don't see type annotation notation as a good fit, either, and yet we have it. Seems to me that DbC would be just as useful as type annotations (and I find D'Aprano's example syntax very readable).
For design by contract, as others have noted Python assert statements work fine for simple preconditions and postconditions.
And print statements work fine for simple debugging. New features are not added for the simple cases, but the complex, or non-obviously correct, or the very useful cases.
I can see where that could get expensive quickly.
Well, we don't have to have exactly the same kind of DbC as Eiffel does. -- ~Ethan~
Hi, @David Mertz, Eric Fahlgren re inheritance: thank you very much for your suggestions. I will try to see how inheritance can be implemented with metaclasses and annotations and put it into icontract library. @David Mertz re costs:
Adding a new feature, even if it is *technically* backwards compatible, has HUGE costs.
[...]
I am also aware of the costs; when I wrote "useful", I actually meant "useful and meaningful to implement given the costs that you mention. Please apologize for my inexact wording (I write these emails in the evening after work). Related to the text I emphasized, would you mind to explain a bit more in-depth which features you have in mind? I see contracts formally written out and automatically verified as a completely indispensable tool in large projects with multiple people involved. As I already outlined, writing them in documentation leads to code rot. Not writing assumptions about the data structures at all causes, in my opinion, tons of problems and very, very slow development since each programmer working on the legacy code needs to figure these assumptions over and over again. While testing is not substitute for thinking, I possible don't see how you can add/refactor code in big projects and consider all the interactions between the components. Unit testing can get you only as far, since you always test only for a concrete case. Contracts get you much further, not to mention the automatically generated tests which I can't conceive practical without some sort of contracts in the code. I have also to admit that I was schooled with contracts so I might have a distorted view on the matter (Bertrand Meyer taught Introduction to Programming and all the software engineering university classes; I also worked as a teaching assistant with him for the Software Architecture class). I will try to see how far we can implement contracts as a library. Personally, I would prefer dedicated syntax and I do believe that dedicated syntax merits the costs since these constructs would lead to much better programming in the Python world. If we can't get new syntax, having contracts in the standard library would do it for me as a compromise -- it would standardize, even if in ugly way (as Steven D'Aprano pointed out), how we deal with contracts in Python and would allow for integration into IDEs and static analysis tools. On Thu, 30 Aug 2018 at 05:01, Ethan Furman <ethan@stoneleaf.us> wrote:
On Thu, Aug 30, 2018 at 3:44 AM Marko Ristin-Kaufmann < marko.ristin@gmail.com> wrote:
This much is rather self-evidently untrue. There are hundreds or thousands of large projects involving multiple people, written in Python, that have succeeded without using contracts. That suggests the feature is not "completely indispensable" since it was dispensed with in the vast majority of large projects. I think that most of these large projects succeed in large part because they have good *unit tests*. There are several popular frameworks for writing these (some in standard library), but notably none of them require specific syntax changes to make them work. Some of them *do* use DSLs of sorts as part of how they operate (or various metaprogramming and introspection magic). There is a whole lot of overlap between what unit tests do and what design-by-contract does, enough so that I believe the latter adds little to a large project (of course you can come up with some specific example that a unit test cannot verify as well as a pre/postcondition. In writing before about "features" (and Paul Moore) does a better job than me in writing about *costs*, I wasn't discussing design-by-contract specifically. Various new feature ideas come up here and elsewhere. A few are accepted, most are rejected. They all need to be compared to costs like those I mention before their possible advantages can win out. Yours, David... -- Keeping medicines from the bloodstreams of the sick; food from the bellies of the hungry; books from the hands of the uneducated; technology from the underdeveloped; and putting advocates of freedom in prisons. Intellectual property is to the 21st century what the slave trade was to the 16th.
Hi, @David Mertz, @Paul Moore: first of all, thank you very much for your thorough answers! @Paul Moore <p.f.moore@gmail.com>: Thanks in particular for suggesting a road map. I still think that we need somewhat broader agreement even on this list that contracts are useful at an abstract level before we dig in and find concrete examples in existing code bases, have a more detailed estimation about the costs and how much conflicts the new keyword would cause, analyze how hard it would be to teach the concepts *etc.* So far, my impression is that the majority rejects outright the idea that the design-by-contract is useful and actually sees it as a redundant approach to software correctness at best, and an useless concept at worst. Of course, it would be much more convincing if I already performed a much more thorough analysis as you suggested. Unfortunately, I don't have the eloquence nor the resources to accomplish that at this moment. I'd imagine such an analysis to be a collaborative project of multiple interested people. I'd find it a futile effort before we have at least a vague agreement here that design-by-contract would be useful. Please let me address here the concrete points you raised in your message. 1. How exactly do these differ from simple assertions? I don't
The contracts to not apply only to a concrete function, but also extend to inherited methods. Let me make an illustration on a stripped-down example of three classes A, B and C, where B and C inherit from A. class A: # A.some_func operates on sorted lists of integers and # it returns an integer larger than the length of # the input list. @icontract.pre(lambda lst: lst == sorted(lst)) @icontract.post(lambda result: result > len(lst)) def some_func(self, lst: List[int]) -> int: # ... class B(A): # B.some_func inherits contracts from A, but overrides the implementation -> # B.some_func also operates only on sorted lists of integers # and needs to return an integer larger than the length # of the input list. def some_func(self, lst: List[int]) -> int: # ... class C(A): # C.some_func also inherits the contracts from A. # It weakens the precondition: # it operates either on sorted lists OR # the lists that are shorter than 10 elements. # # It strenghthens the postcondition: # It needs to return an integer larger than # the length of the input list AND # the result needs to be divisible by 2. @icontract.post(lambda result: result % 2 == 0) def some_func(self, lst: List[int]) -> int: # ... Assume that the function A.some_func needs sorted integers as input (*e.g.* because it uses binary search). Now, class B inherits from A -- say B.some_func also uses binary search. The things get really interesting in the case of class C and its method some_func: to satisfy polymorphism, C.some_func needs to work in all cases where an instance of A could work as well. However, it can do more -- it can also operate in situations where the input list has few elements, but is not necessarily sorted. This is called "weakening" or "relaxing" of a contract. The "strengthening" or "tightening" of a contract is an analogous concept for the post-conditions. Due to polymorphism, C.some_func needs to return a result that satisfies all the postconditions of its ancestors so that we can safely use it in all the situations where we would also use an instance of the ancestor class. But we can strengthen it in C.some_func: it needs to return the result satisfying the ancestors' postcondition and than be constrained some more. I omitted the invariants for brevity. They behave the same as the post-conditions: C needs to satisfy all its own invariants as well as all the invariants of its ancestor classes (A in this case). 2. Are we talking here about adding checks that would run in
There is no way you can use contracts without slowing down the program (as they still needs to be verified at run-time). People who need the production code to run as fast as possible need to disable the contracts (and assertions) altogether. If contracts were fast to be verified (for example, by in-lining the checks into the function body instead of calling a separate function), the cost of many contracts might be still acceptable in the production code. Mind that there is also a gradation: there might be contracts which are simply always too slow to run in production (*e.g., *check that the input or output is sorted) while others are almost always acceptable (*e.g., *check that the input arguments are positive integers or one argument is smaller than the other). There is usually a switch mechanism that allows the developer to turn off certain contracts while still enforcing the others. Please mind that the benefits of the contracts are threefold and are not only reduced to "better assertions": * They allow us to formally write down the assumptions about the input and output of the functions and invariants of the data structures. * They verify these assumptions automatically in all possible cases at run-time (be it during the development or during the production). A smaller set of unit tests hence tests a larger portion of the code then if there were no contracts. * They can be used by downstream tools to automatically generate unit tests and perform static analysis. Imagine if you could just click on a class in Pycharm and generate unit tests for all of its methods automatically within couple of seconds. While these might not cover every use case, this feature would allow you to cover many cases with practically zero work. I don't see how such a tool could be developed without a standardized approach to contracts in python (be it a library or a language construct). @David Mertz:
Sorry, David, my bad; English is not my first language. What I meant to say is that I, as in "personally", could not see how I would tackle a large project with a bigger team with a variety of skill levels in efficient manner without a tool to formally write down the assumptions and have them automatically verified. I do understand that achieving large projects is evidently possible without contracts (and many other tools such as static type checks, assertions, and even unit testing). In my case, I would have a much higher overhead working in a team where the assumptions are written in the documentation and never verified since many bugs and obsolete documentation would just go unnoticed. Doing refactoring or adding new features would be hence much more costly for my team and me. Maybe this is also due to the fact that you refer to frameworks when you refer to "large projects" with many experienced and highly skilled contributors. The developers are most probably keen to update the documentation and thoroughly test the framework and also test the assumptions in combinatorial manner. In contrast, I work on a (larger) project where the comments actually become obsolete almost at the moment we wrote them and the testing budget is limited due to time constraints and non-critical application domain. At present, we are not developing a framework used by many other developers, but a solution based on computer vision. We can live with incorrect code, but it's good that it is as correct as it gets. Personally, I view contracts as a way to improve substantially the correctness of the code with very little overhead. In addition, contracts seem to lead to better and more maintainable code since people actually need to reflect more on the assumptions and write them down formally. This is, of course, a completely subjective impression based on my anecdotal experience. Let me see if I can make the inheritance work with meta-programming and extra magical members (__preconditions__, __postconditions__, __invariants__). Let's consider the icontract library a test case, and then see how much more work would it be to bring it into the standard library. @Steve D'Aprano: could you maybe point us to a couple of studies showing that design-by-contract is beneficial in practice? Cheers, Marko On Thu, 30 Aug 2018 at 13:22, David Mertz <mertz@gnosis.cx> wrote:
def example_func(x, y): def __assert_before__(example_func): #implicit, AST-able assertion expressions # ... code def __assert_after__(example_func): # def __assert_after__invariants_02(example_func): # " But these need to be composed / mixed in in MRO order and overridable; more like a class with a __call__() and metaclassery for source-order composition that has to do substring matches for e.g. __assert__*. Blocks of expressions identified with keywords wouldn't be overrideable ('relaxed',); and would need to do fancy AST that replicates existing MRO? On Thursday, August 30, 2018, Ethan Furman <ethan@stoneleaf.us> wrote:
I've just read and article which makes a good case for providing pre-conditions and post-conditions. http://pgbovine.net/python-unreadable.htm The main point is: "without proper comments and documentation, even the cleanest Python code is incomprehensible 'in-the-large'." I find the article to be thoughtful and well-written. By the way the author, Philip J Guo, is also the author of http://pgbovine.net/publications/non-native-english-speakers-learning-progra... http://pythontutor.com/ I recommend all of the above. -- Jonathan
Jonathan Fine wrote:
There's nothing in there that talks about PBC-style executable preconditions and postconditions, it's all about documenting the large-scale intent and purpose of code. He doesn't put forward any argument why executable code should be a better way to do that than writing comments. Personally I don't think it is. E.g. def distim(doshes): for d in doshes: assert isinstance(d, Dosh) # do something here for d in doshes: assert is_distimmed(d) This ticks the precondition and postcondition boxes, but still doesn't give you any idea what a Dosh is and why you would want to distim it. -- Greg
On Mon, 3 Sep 2018 at 23:51, Greg Ewing <greg.ewing@canterbury.ac.nz> wrote:
FWIW this article looks more like a typical motivational intro to static types in Python :-) (Even his comment about types can be partially answered with e.g. Protocols.) -- Ivan
On Tue, Sep 04, 2018 at 12:08:31AM +0100, Ivan Levkivskyi wrote:
Did we read the same article? This is no more about static typing than it is about contracts. It is about the need for documentation. The only connection here between either static typing or contracts is that both can be a form of very limited documentation: type declarations tell you the types of parameters and variables (but not what range of values they can take or what they represent) and contracts tell you the types and values (but not what they represent). The author explicitly states that statically typed languages have the same problem communicating the meaning of the program. Neither clean syntax (like Python) nor static types help the reader comprehend *what* the program is doing "in the large". He says: What's contained within allCovData and covMap (which I presume are both dicts)? What are the types of the keys? What are the types of the values? More importantly, what is the meaning of the keys, values, and their mapping? What do these objects represent in the grand scheme of the entire program, and how can I best leverage them to do what I want to do? Unfortunately, nothing short of having the programmer write high-level comments and/or personally explain the code to me can possibly provide me with such knowledge. It's not Python's fault, though; I would've faced the same comprehension barriers with analogous code written in Java or C++. Nothing is wrong with Python in this regard, but unfortunately its clear syntax cannot provide any advantages for me when trying to understand code 'in-the-large'. and later goes on to say: (To be fair, static types aren't a panacea either: If I showed you the same Java code filled with type definitions, then it would be easier to understand what this function is doing in terms of its concrete types, but without comments, you still won't be able to understand what this function is doing in terms of its actual underlying purpose, which inevitably involves programmer-intended 'abstract types'.) -- Steve
On Tue, Sep 04, 2018 at 10:50:27AM +1200, Greg Ewing wrote:
Indeed. Apart from a throw-away comment about using pre-conditions and post-conditions, that post has little or nothing to do with contracts specifically.
That's because the article isn't about executable comments versus dumb comments. Its about the need for writing documentation and comments of any sort, so long as it helps people understand the purpose of the code, what and why it does what it does. If you read the author's other posts, he discusses the advantages of assertions over dumb comments in other places, such as here: http://pgbovine.net/programming-with-asserts.htm As far as dumb comments go, I'm reminded of this quote: "At Resolver we've found it useful to short-circuit any doubt and just refer to comments in code as 'lies'." --Michael Foord paraphrases Christian Muirhead on python-dev, 2009-03-22 But you're right that assertions can only give you limited assistence in understanding the large scale structure of code: - types tell you only what kind of thing a variable is; - assertions tell you both the kind of thing and the acceptible values it can take; - unlike dumb comments, assertions are checked, so they stay relevant longer and are less likely to become lies. These can help the reader understand the what and sometimes the how, but to understand the why you need to either be able to infer it from the code, or documentation (including comments). Given the choice between a comment and an assertion: # x must be between 11 and 17 assert 11 <= x <= 17 I think it should be obvious why the assertion is better. But neither explain *why* x must be within that range. Unless it is obvious from context (and often it is!) there should be a reason given, otherwise the reader has to just take it on faith.
I think the author would agree with you 100%, given that his article is talking about the need to understand the *why* of code, not just what it does in small detail, but the large scale reasons for it. -- Steve
On 08/29/2018 04:53 PM, Hugh Fisher wrote:
From: Marko Ristin-Kaufmann:
I don't see type annotation notation as a good fit, either, and yet we have it. Seems to me that DbC would be just as useful as type annotations (and I find D'Aprano's example syntax very readable).
For design by contract, as others have noted Python assert statements work fine for simple preconditions and postconditions.
And print statements work fine for simple debugging. New features are not added for the simple cases, but the complex, or non-obviously correct, or the very useful cases.
I can see where that could get expensive quickly.
Well, we don't have to have exactly the same kind of DbC as Eiffel does. -- ~Ethan~
Hi, @David Mertz, Eric Fahlgren re inheritance: thank you very much for your suggestions. I will try to see how inheritance can be implemented with metaclasses and annotations and put it into icontract library. @David Mertz re costs:
Adding a new feature, even if it is *technically* backwards compatible, has HUGE costs.
[...]
I am also aware of the costs; when I wrote "useful", I actually meant "useful and meaningful to implement given the costs that you mention. Please apologize for my inexact wording (I write these emails in the evening after work). Related to the text I emphasized, would you mind to explain a bit more in-depth which features you have in mind? I see contracts formally written out and automatically verified as a completely indispensable tool in large projects with multiple people involved. As I already outlined, writing them in documentation leads to code rot. Not writing assumptions about the data structures at all causes, in my opinion, tons of problems and very, very slow development since each programmer working on the legacy code needs to figure these assumptions over and over again. While testing is not substitute for thinking, I possible don't see how you can add/refactor code in big projects and consider all the interactions between the components. Unit testing can get you only as far, since you always test only for a concrete case. Contracts get you much further, not to mention the automatically generated tests which I can't conceive practical without some sort of contracts in the code. I have also to admit that I was schooled with contracts so I might have a distorted view on the matter (Bertrand Meyer taught Introduction to Programming and all the software engineering university classes; I also worked as a teaching assistant with him for the Software Architecture class). I will try to see how far we can implement contracts as a library. Personally, I would prefer dedicated syntax and I do believe that dedicated syntax merits the costs since these constructs would lead to much better programming in the Python world. If we can't get new syntax, having contracts in the standard library would do it for me as a compromise -- it would standardize, even if in ugly way (as Steven D'Aprano pointed out), how we deal with contracts in Python and would allow for integration into IDEs and static analysis tools. On Thu, 30 Aug 2018 at 05:01, Ethan Furman <ethan@stoneleaf.us> wrote:
On Thu, Aug 30, 2018 at 3:44 AM Marko Ristin-Kaufmann < marko.ristin@gmail.com> wrote:
This much is rather self-evidently untrue. There are hundreds or thousands of large projects involving multiple people, written in Python, that have succeeded without using contracts. That suggests the feature is not "completely indispensable" since it was dispensed with in the vast majority of large projects. I think that most of these large projects succeed in large part because they have good *unit tests*. There are several popular frameworks for writing these (some in standard library), but notably none of them require specific syntax changes to make them work. Some of them *do* use DSLs of sorts as part of how they operate (or various metaprogramming and introspection magic). There is a whole lot of overlap between what unit tests do and what design-by-contract does, enough so that I believe the latter adds little to a large project (of course you can come up with some specific example that a unit test cannot verify as well as a pre/postcondition. In writing before about "features" (and Paul Moore) does a better job than me in writing about *costs*, I wasn't discussing design-by-contract specifically. Various new feature ideas come up here and elsewhere. A few are accepted, most are rejected. They all need to be compared to costs like those I mention before their possible advantages can win out. Yours, David... -- Keeping medicines from the bloodstreams of the sick; food from the bellies of the hungry; books from the hands of the uneducated; technology from the underdeveloped; and putting advocates of freedom in prisons. Intellectual property is to the 21st century what the slave trade was to the 16th.
Hi, @David Mertz, @Paul Moore: first of all, thank you very much for your thorough answers! @Paul Moore <p.f.moore@gmail.com>: Thanks in particular for suggesting a road map. I still think that we need somewhat broader agreement even on this list that contracts are useful at an abstract level before we dig in and find concrete examples in existing code bases, have a more detailed estimation about the costs and how much conflicts the new keyword would cause, analyze how hard it would be to teach the concepts *etc.* So far, my impression is that the majority rejects outright the idea that the design-by-contract is useful and actually sees it as a redundant approach to software correctness at best, and an useless concept at worst. Of course, it would be much more convincing if I already performed a much more thorough analysis as you suggested. Unfortunately, I don't have the eloquence nor the resources to accomplish that at this moment. I'd imagine such an analysis to be a collaborative project of multiple interested people. I'd find it a futile effort before we have at least a vague agreement here that design-by-contract would be useful. Please let me address here the concrete points you raised in your message. 1. How exactly do these differ from simple assertions? I don't
The contracts to not apply only to a concrete function, but also extend to inherited methods. Let me make an illustration on a stripped-down example of three classes A, B and C, where B and C inherit from A. class A: # A.some_func operates on sorted lists of integers and # it returns an integer larger than the length of # the input list. @icontract.pre(lambda lst: lst == sorted(lst)) @icontract.post(lambda result: result > len(lst)) def some_func(self, lst: List[int]) -> int: # ... class B(A): # B.some_func inherits contracts from A, but overrides the implementation -> # B.some_func also operates only on sorted lists of integers # and needs to return an integer larger than the length # of the input list. def some_func(self, lst: List[int]) -> int: # ... class C(A): # C.some_func also inherits the contracts from A. # It weakens the precondition: # it operates either on sorted lists OR # the lists that are shorter than 10 elements. # # It strenghthens the postcondition: # It needs to return an integer larger than # the length of the input list AND # the result needs to be divisible by 2. @icontract.post(lambda result: result % 2 == 0) def some_func(self, lst: List[int]) -> int: # ... Assume that the function A.some_func needs sorted integers as input (*e.g.* because it uses binary search). Now, class B inherits from A -- say B.some_func also uses binary search. The things get really interesting in the case of class C and its method some_func: to satisfy polymorphism, C.some_func needs to work in all cases where an instance of A could work as well. However, it can do more -- it can also operate in situations where the input list has few elements, but is not necessarily sorted. This is called "weakening" or "relaxing" of a contract. The "strengthening" or "tightening" of a contract is an analogous concept for the post-conditions. Due to polymorphism, C.some_func needs to return a result that satisfies all the postconditions of its ancestors so that we can safely use it in all the situations where we would also use an instance of the ancestor class. But we can strengthen it in C.some_func: it needs to return the result satisfying the ancestors' postcondition and than be constrained some more. I omitted the invariants for brevity. They behave the same as the post-conditions: C needs to satisfy all its own invariants as well as all the invariants of its ancestor classes (A in this case). 2. Are we talking here about adding checks that would run in
There is no way you can use contracts without slowing down the program (as they still needs to be verified at run-time). People who need the production code to run as fast as possible need to disable the contracts (and assertions) altogether. If contracts were fast to be verified (for example, by in-lining the checks into the function body instead of calling a separate function), the cost of many contracts might be still acceptable in the production code. Mind that there is also a gradation: there might be contracts which are simply always too slow to run in production (*e.g., *check that the input or output is sorted) while others are almost always acceptable (*e.g., *check that the input arguments are positive integers or one argument is smaller than the other). There is usually a switch mechanism that allows the developer to turn off certain contracts while still enforcing the others. Please mind that the benefits of the contracts are threefold and are not only reduced to "better assertions": * They allow us to formally write down the assumptions about the input and output of the functions and invariants of the data structures. * They verify these assumptions automatically in all possible cases at run-time (be it during the development or during the production). A smaller set of unit tests hence tests a larger portion of the code then if there were no contracts. * They can be used by downstream tools to automatically generate unit tests and perform static analysis. Imagine if you could just click on a class in Pycharm and generate unit tests for all of its methods automatically within couple of seconds. While these might not cover every use case, this feature would allow you to cover many cases with practically zero work. I don't see how such a tool could be developed without a standardized approach to contracts in python (be it a library or a language construct). @David Mertz:
Sorry, David, my bad; English is not my first language. What I meant to say is that I, as in "personally", could not see how I would tackle a large project with a bigger team with a variety of skill levels in efficient manner without a tool to formally write down the assumptions and have them automatically verified. I do understand that achieving large projects is evidently possible without contracts (and many other tools such as static type checks, assertions, and even unit testing). In my case, I would have a much higher overhead working in a team where the assumptions are written in the documentation and never verified since many bugs and obsolete documentation would just go unnoticed. Doing refactoring or adding new features would be hence much more costly for my team and me. Maybe this is also due to the fact that you refer to frameworks when you refer to "large projects" with many experienced and highly skilled contributors. The developers are most probably keen to update the documentation and thoroughly test the framework and also test the assumptions in combinatorial manner. In contrast, I work on a (larger) project where the comments actually become obsolete almost at the moment we wrote them and the testing budget is limited due to time constraints and non-critical application domain. At present, we are not developing a framework used by many other developers, but a solution based on computer vision. We can live with incorrect code, but it's good that it is as correct as it gets. Personally, I view contracts as a way to improve substantially the correctness of the code with very little overhead. In addition, contracts seem to lead to better and more maintainable code since people actually need to reflect more on the assumptions and write them down formally. This is, of course, a completely subjective impression based on my anecdotal experience. Let me see if I can make the inheritance work with meta-programming and extra magical members (__preconditions__, __postconditions__, __invariants__). Let's consider the icontract library a test case, and then see how much more work would it be to bring it into the standard library. @Steve D'Aprano: could you maybe point us to a couple of studies showing that design-by-contract is beneficial in practice? Cheers, Marko On Thu, 30 Aug 2018 at 13:22, David Mertz <mertz@gnosis.cx> wrote:
def example_func(x, y): def __assert_before__(example_func): #implicit, AST-able assertion expressions # ... code def __assert_after__(example_func): # def __assert_after__invariants_02(example_func): # " But these need to be composed / mixed in in MRO order and overridable; more like a class with a __call__() and metaclassery for source-order composition that has to do substring matches for e.g. __assert__*. Blocks of expressions identified with keywords wouldn't be overrideable ('relaxed',); and would need to do fancy AST that replicates existing MRO? On Thursday, August 30, 2018, Ethan Furman <ethan@stoneleaf.us> wrote:
I've just read and article which makes a good case for providing pre-conditions and post-conditions. http://pgbovine.net/python-unreadable.htm The main point is: "without proper comments and documentation, even the cleanest Python code is incomprehensible 'in-the-large'." I find the article to be thoughtful and well-written. By the way the author, Philip J Guo, is also the author of http://pgbovine.net/publications/non-native-english-speakers-learning-progra... http://pythontutor.com/ I recommend all of the above. -- Jonathan
Jonathan Fine wrote:
There's nothing in there that talks about PBC-style executable preconditions and postconditions, it's all about documenting the large-scale intent and purpose of code. He doesn't put forward any argument why executable code should be a better way to do that than writing comments. Personally I don't think it is. E.g. def distim(doshes): for d in doshes: assert isinstance(d, Dosh) # do something here for d in doshes: assert is_distimmed(d) This ticks the precondition and postcondition boxes, but still doesn't give you any idea what a Dosh is and why you would want to distim it. -- Greg
On Mon, 3 Sep 2018 at 23:51, Greg Ewing <greg.ewing@canterbury.ac.nz> wrote:
FWIW this article looks more like a typical motivational intro to static types in Python :-) (Even his comment about types can be partially answered with e.g. Protocols.) -- Ivan
On Tue, Sep 04, 2018 at 12:08:31AM +0100, Ivan Levkivskyi wrote:
Did we read the same article? This is no more about static typing than it is about contracts. It is about the need for documentation. The only connection here between either static typing or contracts is that both can be a form of very limited documentation: type declarations tell you the types of parameters and variables (but not what range of values they can take or what they represent) and contracts tell you the types and values (but not what they represent). The author explicitly states that statically typed languages have the same problem communicating the meaning of the program. Neither clean syntax (like Python) nor static types help the reader comprehend *what* the program is doing "in the large". He says: What's contained within allCovData and covMap (which I presume are both dicts)? What are the types of the keys? What are the types of the values? More importantly, what is the meaning of the keys, values, and their mapping? What do these objects represent in the grand scheme of the entire program, and how can I best leverage them to do what I want to do? Unfortunately, nothing short of having the programmer write high-level comments and/or personally explain the code to me can possibly provide me with such knowledge. It's not Python's fault, though; I would've faced the same comprehension barriers with analogous code written in Java or C++. Nothing is wrong with Python in this regard, but unfortunately its clear syntax cannot provide any advantages for me when trying to understand code 'in-the-large'. and later goes on to say: (To be fair, static types aren't a panacea either: If I showed you the same Java code filled with type definitions, then it would be easier to understand what this function is doing in terms of its concrete types, but without comments, you still won't be able to understand what this function is doing in terms of its actual underlying purpose, which inevitably involves programmer-intended 'abstract types'.) -- Steve
On Tue, Sep 04, 2018 at 10:50:27AM +1200, Greg Ewing wrote:
Indeed. Apart from a throw-away comment about using pre-conditions and post-conditions, that post has little or nothing to do with contracts specifically.
That's because the article isn't about executable comments versus dumb comments. Its about the need for writing documentation and comments of any sort, so long as it helps people understand the purpose of the code, what and why it does what it does. If you read the author's other posts, he discusses the advantages of assertions over dumb comments in other places, such as here: http://pgbovine.net/programming-with-asserts.htm As far as dumb comments go, I'm reminded of this quote: "At Resolver we've found it useful to short-circuit any doubt and just refer to comments in code as 'lies'." --Michael Foord paraphrases Christian Muirhead on python-dev, 2009-03-22 But you're right that assertions can only give you limited assistence in understanding the large scale structure of code: - types tell you only what kind of thing a variable is; - assertions tell you both the kind of thing and the acceptible values it can take; - unlike dumb comments, assertions are checked, so they stay relevant longer and are less likely to become lies. These can help the reader understand the what and sometimes the how, but to understand the why you need to either be able to infer it from the code, or documentation (including comments). Given the choice between a comment and an assertion: # x must be between 11 and 17 assert 11 <= x <= 17 I think it should be obvious why the assertion is better. But neither explain *why* x must be within that range. Unless it is obvious from context (and often it is!) there should be a reason given, otherwise the reader has to just take it on faith.
I think the author would agree with you 100%, given that his article is talking about the need to understand the *why* of code, not just what it does in small detail, but the large scale reasons for it. -- Steve
participants (9)
-
David Mertz -
Ethan Furman -
Greg Ewing -
Hugh Fisher -
Ivan Levkivskyi -
Jonathan Fine -
Marko Ristin-Kaufmann -
Steven D'Aprano -
Wes Turner