<div dir="ltr"><div dir="ltr"><div dir="ltr"><div dir="ltr"><div dir="ltr"><div dir="ltr"><div><div><div><div>Hi,<br><br></div>@David Mertz, @Paul Moore: first of all, thank you very much for your thorough answers!<br><br></div><a class="gmail_plusreply" id="gmail-plusReplyChip-1" href="mailto:p.f.moore@gmail.com" tabindex="-1">@Paul Moore</a>: Thanks in particular for suggesting a road map. I still think that we need somewhat broader agreement even on this list that contracts are useful at an abstract level before we dig in and find concrete examples in existing code bases, have a more detailed estimation about the costs and how much conflicts the new keyword would cause, analyze how hard it would be to teach the concepts <i>etc.</i> So far, my impression is that the majority rejects outright the idea that the design-by-contract is useful and actually sees it as a redundant approach to software correctness at best, and an useless concept at worst. <br><br></div><div>Of course, it would be much more convincing if I already performed a much more thorough analysis as you suggested. Unfortunately, I don't have the eloquence nor the resources to accomplish that at this moment. I'd imagine such an analysis to be a collaborative project of multiple interested people. I'd find it a futile effort before we have at least a vague agreement here that design-by-contract would be useful.<br></div><div><br></div>Please let me address here the concrete points you raised in your message.<br><br><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex">1. How exactly do these differ from simple assertions? I don't<br>
understand the hints about "relaxing" and "strengthening" contracts in<br>
subclasses, in particular I've no idea how I'd express that in actual<br>
syntax.<br></blockquote><div><br></div><div>The contracts to not apply only to a concrete function, but also extend to inherited methods.<br><br></div><div>Let me make an<font size="2"> illustration on a stripped-down example of three classes A, B and C, where B and C inherit from A.<br></font><pre style="background-color:rgb(255,255,255);color:rgb(0,0,0);font-family:"DejaVu Sans Mono""><span style="color:rgb(0,0,128);font-weight:bold">class </span>A:<br> <span style="color:rgb(128,128,128);font-style:italic"># A.some_func operates on sorted lists of integers and <br></span><span style="color:rgb(128,128,128);font-style:italic"> # it returns an integer larger than the length of <br></span><span style="color:rgb(128,128,128);font-style:italic"> # the input list.<br></span><span style="color:rgb(128,128,128);font-style:italic"> </span><span style="color:rgb(0,0,178)">@icontract.pre</span>(<span style="color:rgb(0,0,128);font-weight:bold">lambda </span>lst: lst == <span style="color:rgb(0,0,128)">sorted</span>(lst))<br> <span style="color:rgb(0,0,178)">@icontract.post</span>(<span style="color:rgb(0,0,128);font-weight:bold">lambda </span>result: result > <span style="color:rgb(0,0,128)">len</span>(lst))<br> <span style="color:rgb(0,0,128);font-weight:bold">def </span>some_func(<span style="color:rgb(148,85,141)">self</span>, lst: List[<span style="color:rgb(0,0,128)">int</span>]) -> <span style="color:rgb(0,0,128)">int</span>:<br> <span style="color:rgb(128,128,128);font-style:italic"># ...<br></span><span style="color:rgb(128,128,128);font-style:italic"><br></span><span style="color:rgb(0,0,128);font-weight:bold">class </span>B(A):<br> <span style="color:rgb(128,128,128);font-style:italic"># B.some_func inherits contracts from A, but overrides the implementation -><br></span><span style="color:rgb(128,128,128);font-style:italic"> # B.some_func also operates only on sorted lists of integers<br></span><span style="color:rgb(128,128,128);font-style:italic"> # and needs to return an integer larger than the length<br></span><span style="color:rgb(128,128,128);font-style:italic"> # of the input list.<br></span><span style="color:rgb(128,128,128);font-style:italic"> </span><span style="color:rgb(0,0,128);font-weight:bold">def </span>some_func(<span style="color:rgb(148,85,141)">self</span>, lst: List[<span style="color:rgb(0,0,128)">int</span>]) -> <span style="color:rgb(0,0,128)">int</span>:<br> <span style="color:rgb(128,128,128);font-style:italic"># ...<br></span><span style="color:rgb(128,128,128);font-style:italic"><br></span><span style="color:rgb(128,128,128);font-style:italic"><br></span><span style="color:rgb(0,0,128);font-weight:bold">class </span>C(A):<br> <span style="color:rgb(128,128,128);font-style:italic"># C.some_func also inherits the contracts from A.<br></span><span style="color:rgb(128,128,128);font-style:italic"> # It weakens the precondition:<br></span><span style="color:rgb(128,128,128);font-style:italic"> # it operates either on sorted lists OR<br></span><span style="color:rgb(128,128,128);font-style:italic"> # the lists that are shorter than 10 elements.<br></span><span style="color:rgb(128,128,128);font-style:italic"> #<br></span><span style="color:rgb(128,128,128);font-style:italic"> # It strenghthens the postcondition:<br></span><span style="color:rgb(128,128,128);font-style:italic"> # It needs to return an integer larger than <br></span><span style="color:rgb(128,128,128);font-style:italic"> # the length of the input list AND<br></span><span style="color:rgb(128,128,128);font-style:italic"> # the result needs to be divisible by 2.</span><br> <span style="color:rgb(0,0,178)">@icontract.post</span>(<span style="color:rgb(0,0,128);font-weight:bold">lambda </span>result: result % <span style="color:rgb(0,0,255)">2 </span>== <span style="color:rgb(0,0,255)">0</span>)<br> <span style="color:rgb(0,0,128);font-weight:bold">def </span>some_func(<span style="color:rgb(148,85,141)">self</span>, lst: List[<span style="color:rgb(0,0,128)">int</span>]) -> <span style="color:rgb(0,0,128)">int</span>:<br> <span style="color:rgb(128,128,128);font-style:italic"># ...</span></pre></div><div>Assume that the function A.some_func needs sorted integers as input (<i>e.g.</i> because it uses binary search). <br><br></div><div>Now, class B inherits from A -- say B.some_func also uses binary search.<br><br></div><div>The things get really interesting in the case of class C and its method some_func: to satisfy polymorphism, C.some_func needs to work in all cases where an instance of A could work as well. However, it can do more -- it can also operate in situations where the input list has few elements, but is not necessarily sorted. This is called "weakening" or "relaxing" of a contract.<br><br></div><div>The "strengthening" or "tightening" of a contract is an analogous concept for the post-conditions. Due to polymorphism, C.some_func needs to return a result that satisfies all the postconditions of its ancestors so that we can safely use it in all the situations where we would also use an instance of the ancestor class. But we can strengthen it in C.some_func: it needs to return the result satisfying the ancestors' postcondition and than be constrained some more.<br><br></div><div>I omitted the invariants for brevity. They behave the same as the post-conditions: C needs to satisfy all its own invariants as well as all the invariants of its ancestor classes (A in this case).<br></div><div><br></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex">
2. Are we talking here about adding checks that would run in<br>
production code? Wouldn't that slow code down? How do I deal with the<br>
conflict between wanting tighter checks but not wanting to pay the<br>
cost at runtime of checking things that should never go wrong<br>
(contracts, as I understand it, are for protecting against code bugs,<br>
that's why there's no facility for trapping them and producing "user<br>
friendly" error messages)? I use assertions very sparingly in<br>
production code for precisely that reason - why would I be more<br>
willing to use contracts?</blockquote><br></div><div>There is no way you can use contracts without slowing down the program (as they still needs to be verified at run-time). People who need the production code to run as fast as possible need to disable the contracts (and assertions) altogether. If contracts were fast to be verified (for example, by in-lining the checks into the function body instead of calling a separate function), the cost of many contracts might be still acceptable in the production code. Mind that there is also a gradation: there might be contracts which are simply always too slow to run in production (<i>e.g., </i>check that the input or output is sorted) while others are almost always acceptable (<i>e.g., </i>check that the input arguments are positive integers or one argument is smaller than the other). There is usually a switch mechanism that allows the developer to turn off certain contracts while still enforcing the others.<br><br></div><div>Please mind that the benefits of the contracts are threefold and are not only reduced to "better assertions":<br></div><div>* They allow us to formally write down the assumptions about the input and output of the functions and invariants of the data structures.<br></div><div>* They verify these assumptions automatically in all possible cases at run-time (be it during the development or during the production). A smaller set of unit tests hence tests a larger portion of the code then if there were no contracts.<br></div><div>* They can be used by downstream tools to automatically generate unit tests and perform static analysis. Imagine if you could just click on a class in Pycharm and generate unit tests for all of its methods automatically within couple of seconds. While these might not cover every use case, this feature would allow you to cover many cases with practically zero work. I don't see how such a tool could be developed without a standardized approach to contracts in python (be it a library or a language construct).<br><br></div><div>@David Mertz:<br><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><span class="gmail-im"><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div dir="ltr"><div>Related
to the text I emphasized, would you mind to explain a bit more in-depth
which features you have in mind? I see contracts formally written out
and automatically verified as a completely indispensable tool in large
projects with multiple people involved. </div></div></blockquote><div><br></div></span><div>This
much is rather self-evidently untrue. There are hundreds or thousands
of large projects involving multiple people, written in Python, that
have succeeded without using contracts. That suggests the feature is
not "completely indispensable" since it was dispensed with in the vast
majority of large projects.</div></blockquote></div><div><br></div><div>Sorry, David, my bad; English is not my first language. What I meant to say is that I, as in "personally", could not see how I would tackle a large project with a bigger team with a variety of skill levels in efficient manner without a tool to formally write down the assumptions and have them automatically verified. I do understand that achieving large projects is evidently possible without contracts (and many other tools such as static type checks, assertions, and even unit testing). In my case, I would have a much higher overhead working in a team where the assumptions are written in the documentation and never verified since many bugs and obsolete documentation would just go unnoticed. Doing refactoring or adding new features would be hence much more costly for my team and me.<br><br></div><div>Maybe this is also due to the fact that you refer to frameworks when you refer to "large projects" with many experienced and highly skilled contributors. The developers are most probably keen to update the documentation and thoroughly test the framework and also test the assumptions in combinatorial manner. In contrast, I work on a (larger) project where the comments actually become obsolete almost at the moment we wrote them and the testing budget is limited due to time constraints and non-critical application domain. At present, we are not developing a framework used by many other developers, but a solution based on computer vision. We can live with incorrect code, but it's good that it is as correct as it gets.<br><br>Personally, I view contracts as a way to improve substantially the correctness of the code with very little overhead. In addition, contracts seem to lead to better and more maintainable code since people actually need to reflect more on the assumptions and write them down formally. This is, of course, a completely subjective impression based on my anecdotal experience. <br><br></div><div>Let me see if I can make the inheritance work with meta-programming and extra magical members (__preconditions__, __postconditions__, __invariants__). Let's consider the icontract library a test case, and then see how much more work would it be to bring it into the standard library.<br></div><div><br>@Steve D'Aprano: could you maybe point us to a couple of studies showing that design-by-contract is beneficial in practice?<br></div><div><br></div><div>Cheers,<br></div><div>Marko<br></div></div></div></div></div></div><br><div class="gmail_quote"><div dir="ltr">On Thu, 30 Aug 2018 at 13:22, David Mertz <<a href="mailto:mertz@gnosis.cx">mertz@gnosis.cx</a>> wrote:<br></div><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div dir="ltr"><div class="gmail_quote"><div dir="ltr">On Thu, Aug 30, 2018 at 3:44 AM Marko Ristin-Kaufmann <<a href="mailto:marko.ristin@gmail.com" target="_blank">marko.ristin@gmail.com</a>> wrote:</div><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div dir="ltr"><div>Related to the text I emphasized, would you mind to explain a bit more in-depth which features you have in mind? I see contracts formally written out and automatically verified as a completely indispensable tool in large projects with multiple people involved. </div></div></blockquote><div><br></div><div>This much is rather self-evidently untrue. There are hundreds or thousands of large projects involving multiple people, written in Python, that have succeeded without using contracts. That suggests the feature is not "completely indispensable" since it was dispensed with in the vast majority of large projects.</div><div><br></div><div>I think that most of these large projects succeed in large part because they have good *unit tests*. There are several popular frameworks for writing these (some in standard library), but notably none of them require specific syntax changes to make them work. Some of them *do* use DSLs of sorts as part of how they operate (or various metaprogramming and introspection magic). There is a whole lot of overlap between what unit tests do and what design-by-contract does, enough so that I believe the latter adds little to a large project (of course you can come up with some specific example that a unit test cannot verify as well as a pre/postcondition.</div><div><br></div><div>In writing before about "features" (and Paul Moore) does a better job than me in writing about *costs*, I wasn't discussing design-by-contract specifically. Various new feature ideas come up here and elsewhere. A few are accepted, most are rejected. They all need to be compared to costs like those I mention before their possible advantages can win out.</div><div><br></div><div>Yours, David...</div><div><br></div></div>-- <br><div dir="ltr" class="m_-5051367518524093509gmail_signature" data-smartmail="gmail_signature">Keeping medicines from the bloodstreams of the sick; food <br>from the bellies of the hungry; books from the hands of the <br>uneducated; technology from the underdeveloped; and putting <br>advocates of freedom in prisons. Intellectual property is<br>to the 21st century what the slave trade was to the 16th.<br></div></div>
</blockquote></div></div>