Why is design-by-contracts not widely adopted?
Hi, (I'd like to fork from a previous thread, "Pre-conditions and post-conditions", since it got long and we started discussing a couple of different things. Let's put the general discussion related to design-by-contract in this thread and I'll spawn another thread for the discussion about the concrete implementation of a design-by-contract library in Python.) After the discussion we had on the list and after browsing the internet a bit, I'm still puzzled why design-by-contract was not more widely adopted and why so few languages support it. Please have a look at these articles and answers: - https://www.leadingagile.com/2018/05/design-by-contract-part-one/ - https://ask.slashdot.org/story/07/03/10/009237/why-is-design-by-contract-not... -- this one is from 2007, but represents well IMO the way people discuss it - https://stackoverflow.com/questions/481312/why-is-design-by-contract-not-so-... and this answer in particular https://stackoverflow.com/a/28680756/1600678 - https://softwareengineering.stackexchange.com/questions/128717/why-is-there-... I did see that there are a lot of misconceptions about it ("simple asserts", "developer overhead", "needs upfront design", "same as unit testing"). This is probably the case with any novel concept that people are not familiar with. However, what does puzzle me is that once the misconceptions are rectified ("it's not simple asserts", "the development is actually faster", "no need for upfront design", "not orthogonal, but dbc + unit testing is better than just unit testing"), the concept is still discarded *. *After properly reading about design-by-contract and getting deeper into the topic, there is no rational argument against it and the benefits are obvious. And still, people just wave their hand and continue without formalizing the contracts in the code and keep on writing them in the descriptions. * Why is that so? *I'm completely at loss about that -- especially about the historical reasons (some mentioned that design-by-contract did not take off since Bertrand Meyer holds the trademark on the term and because of his character. Is that the reason?). One explanation that seems plausible to me is that many programmers are actually having a hard time with formalization and logic rules (*e.g., *implication, quantifiers), maybe due to missing education (*e.g. *many programmers are people who came to programming from other less-formal fields). It's hence easier for them to write in human text and takes substantial cognitive load to formalize these thoughts in code. Does that explains it? What do you think? What is the missing part of the puzzle? Cheers, Marko
On Sun, Sep 23, 2018, 1:10 AM Marko Ristin-Kaufmann <marko.ristin@gmail.com> wrote:
I've tried to explain my own reasons for not being that interested in DbC in other threads. I've been familiar with DbC libraries in Python for close to 20 years, and it never struck me as worth the effort of using. I'm not alone in this. A large majority of folks formally educted in computer science and related fields have been aware of DbC for decades but deliberately decided not to use them in their own code. Maybe you and Bertram Meyer are simple better than that 99% of programmers... Or maybe the benefit is not so self-evidently and compelling as it feels to you. To me, as I've said, DbC imposes a very large cost for both writers and readers of code. While it's possible to split hairs about the edge cases where assertions and unit tests cannot cover identical ground, the reality is that the benefits are extremely close between the different techniques. However, it's vastly easier to take a more incremental and as-needed approach using assertions and unit tests than it is with DbC. Moreover, unit tests have the giant advantage of living *elsewhere* than in the main code itself... This probably doesn't matter so much to writers, but it's a huge win for readers. Even with doctests—which I'm somewhat unusual in actually liking—even though the tests live in the same file and function/class as the operational code, it still feels relatively easy to separate the concerns visual when reading such code. I just cannot get that with DbC. I know you can inside I'm wrong about all this, and my code would be better and faster if I would accept this niche orthodoxy. But I just do not see DbC becoming non-niche in any plausible future, neither in Python not in any other mainstream language. Opinions are free to differ, and I could be wrong.
On Sun, Sep 23, 2018 at 07:09:37AM +0200, Marko Ristin-Kaufmann wrote:
You are asking a question about human psychology but expecting logical, technical answers. I think that's doomed to failure. There is no nice way to say this, because it isn't nice. Programmers and language designers don't use DbC because it is new and different and therefore scary and wrong. Programmers, as a class, are lazy (they even call laziness a virtue), deeply conservative, superstitious, and don't like change. They do what they do because they're used to it, not because it is the technically correct thing to do, unless it is by accident. (Before people's hackles raise too much, I'm speaking in generalities, not about you personally. Of course you are one of the 1% awesomely logical, technically correct programmers who know what you are doing and do it for the right reasons after careful analysis. I'm talking about the other 99%, you know the ones. You probably work with them. You've certainly read their answers on Stackoverflow or The Daily WTF and a million blogs.) They won't use it until there is a critical mass of people using it, and then it will suddenly flip from "that weird shit Eiffel does" to "oh yeah, this is a standard technique that everyone uses, although we don't make a big deal about it". Every innovation in programming goes through this. Whether the innovation goes mainstream or not depends on getting a critical mass, and that is all about psychology and network effects and nothing to do with the merit of the idea. Remember the wars over structured programming? Probably not. In 2018, the idea that people would *seriously argue against writing subroutines* seems insane, but they did. As late as 1999, a former acquaintance of mine was working on a BASIC project for a manager who insisted they use GOTO and GOSUB in preference to subroutines. Testing: the idea that we should have *repeatable automated tests* took a long time to be accepted, and is still resisted by both developers and their managers. What's wrong with just sitting a person down in front of the program and checking for bugs by hand? We still sometimes have to fight for an automated test suite, never mind something like test driven development. ML-based languages have had type inference for decades, and yet people still think of type checking in terms of C and Java declarations. Or they still think in terms of static VERSUS dynamic typing, instead of static PLUS dynamic typing. I could go on, but I think I've made my point. I can give you some technical reasons why I don't use contracts in my own Python code, even though I want to: (1) Many of my programs are too small to bother. If I'm writing a quick script, I don't even write tests. Sometimes "be lazy" is the right answer, when the cost of bugs is small enough and the effort to avoid them is greater. And that's fine. Nobody says that contracts must be mandatory. (2) Python doesn't make it easy to write contracts. None of the solutions I've seen are nice. Ironically, the least worst I've seen is a quick and dirty metaclass solution given by Guido in an essay way back in Python 1.5 days: https://www.python.org/doc/essays/metaclasses/ His solution relies only on a naming convention, no decorators, no lambdas: class C(Eiffel): def method(self, arg): return whatever def method_pre(self, arg): assert arg > 0 def method_post(self, Result, arg): assert Result > arg Still not pretty, but at least we get some block structure instead of a wall of decorators. Syntax matters. Without good syntax that makes it easy to write contracts, it will never be anything but niche. (3) In a sense, *of course I write contracts*. Or at least precondition checks. I just don't call them that, and I embed them in the implementation of the method, and have no way to turn them off. Nearly all of us have done the same, we start with a method like this: class C: def method(self, alist, astring, afloat): # do some work... using nothing but naming conventions and an informal (and often vague) specification in the docstring, and while that is sometimes enough in small projects, the third time we get bitten we start adding defensive checks so we get sensible exceptions: def method(self, alist, astring, afloat): if not isinstance(alist, list): raise TypeError('expected a list') if alist == []: raise ValueError('list must not be empty') # and so on... These are pre-conditions! We just don't call them that. And we can't disable them. They're not easy to extract from the source code and turn into specifications. And so much boilerplate! Let's invent syntax to make it more obvious what is going on: def method(self, alist, astring, afloat): requires: isinstance(alist, list) alist != [] isinstance(astring, str) number_of_vowels(astring) > 0 isinstance(afloat, float) not math.isnan(afloat) implementation: # code goes here Its easy to distinguish the precondition checks from the implementation, easy to ignore the checks if you don't care about them, and easy for an external tool to analyse. What's not to like about contracts? You're already using them, just in an ad hoc, ugly, informal way. And I think that is probably the crux of the matter. Most people are lazy, and don't like having to do things in a systematic manner. For years, programmers resisted writing functions, because unstructured code was easier. We still resist writing documentation, because "its obvious from the source code" is easier. We resist writing even loose specifications, because NOT writing them is easier. We resist writing tests unless the project demands it. We resist running a linter or static checker over our code ("it runs, that means there are no errors"). Until peer pressure and pain makes us do so, then we love them and could not imagine going back to the bad old days before static analysis and linters and unit tests. -- Steve
On Sun, Sep 23, 2018, 1:10 AM Marko Ristin-Kaufmann <marko.ristin@gmail.com> wrote:
I've tried to explain my own reasons for not being that interested in DbC in other threads. I've been familiar with DbC libraries in Python for close to 20 years, and it never struck me as worth the effort of using. I'm not alone in this. A large majority of folks formally educted in computer science and related fields have been aware of DbC for decades but deliberately decided not to use them in their own code. Maybe you and Bertram Meyer are simple better than that 99% of programmers... Or maybe the benefit is not so self-evidently and compelling as it feels to you. To me, as I've said, DbC imposes a very large cost for both writers and readers of code. While it's possible to split hairs about the edge cases where assertions and unit tests cannot cover identical ground, the reality is that the benefits are extremely close between the different techniques. However, it's vastly easier to take a more incremental and as-needed approach using assertions and unit tests than it is with DbC. Moreover, unit tests have the giant advantage of living *elsewhere* than in the main code itself... This probably doesn't matter so much to writers, but it's a huge win for readers. Even with doctests—which I'm somewhat unusual in actually liking—even though the tests live in the same file and function/class as the operational code, it still feels relatively easy to separate the concerns visual when reading such code. I just cannot get that with DbC. I know you can inside I'm wrong about all this, and my code would be better and faster if I would accept this niche orthodoxy. But I just do not see DbC becoming non-niche in any plausible future, neither in Python not in any other mainstream language. Opinions are free to differ, and I could be wrong.
On Sun, Sep 23, 2018 at 07:09:37AM +0200, Marko Ristin-Kaufmann wrote:
You are asking a question about human psychology but expecting logical, technical answers. I think that's doomed to failure. There is no nice way to say this, because it isn't nice. Programmers and language designers don't use DbC because it is new and different and therefore scary and wrong. Programmers, as a class, are lazy (they even call laziness a virtue), deeply conservative, superstitious, and don't like change. They do what they do because they're used to it, not because it is the technically correct thing to do, unless it is by accident. (Before people's hackles raise too much, I'm speaking in generalities, not about you personally. Of course you are one of the 1% awesomely logical, technically correct programmers who know what you are doing and do it for the right reasons after careful analysis. I'm talking about the other 99%, you know the ones. You probably work with them. You've certainly read their answers on Stackoverflow or The Daily WTF and a million blogs.) They won't use it until there is a critical mass of people using it, and then it will suddenly flip from "that weird shit Eiffel does" to "oh yeah, this is a standard technique that everyone uses, although we don't make a big deal about it". Every innovation in programming goes through this. Whether the innovation goes mainstream or not depends on getting a critical mass, and that is all about psychology and network effects and nothing to do with the merit of the idea. Remember the wars over structured programming? Probably not. In 2018, the idea that people would *seriously argue against writing subroutines* seems insane, but they did. As late as 1999, a former acquaintance of mine was working on a BASIC project for a manager who insisted they use GOTO and GOSUB in preference to subroutines. Testing: the idea that we should have *repeatable automated tests* took a long time to be accepted, and is still resisted by both developers and their managers. What's wrong with just sitting a person down in front of the program and checking for bugs by hand? We still sometimes have to fight for an automated test suite, never mind something like test driven development. ML-based languages have had type inference for decades, and yet people still think of type checking in terms of C and Java declarations. Or they still think in terms of static VERSUS dynamic typing, instead of static PLUS dynamic typing. I could go on, but I think I've made my point. I can give you some technical reasons why I don't use contracts in my own Python code, even though I want to: (1) Many of my programs are too small to bother. If I'm writing a quick script, I don't even write tests. Sometimes "be lazy" is the right answer, when the cost of bugs is small enough and the effort to avoid them is greater. And that's fine. Nobody says that contracts must be mandatory. (2) Python doesn't make it easy to write contracts. None of the solutions I've seen are nice. Ironically, the least worst I've seen is a quick and dirty metaclass solution given by Guido in an essay way back in Python 1.5 days: https://www.python.org/doc/essays/metaclasses/ His solution relies only on a naming convention, no decorators, no lambdas: class C(Eiffel): def method(self, arg): return whatever def method_pre(self, arg): assert arg > 0 def method_post(self, Result, arg): assert Result > arg Still not pretty, but at least we get some block structure instead of a wall of decorators. Syntax matters. Without good syntax that makes it easy to write contracts, it will never be anything but niche. (3) In a sense, *of course I write contracts*. Or at least precondition checks. I just don't call them that, and I embed them in the implementation of the method, and have no way to turn them off. Nearly all of us have done the same, we start with a method like this: class C: def method(self, alist, astring, afloat): # do some work... using nothing but naming conventions and an informal (and often vague) specification in the docstring, and while that is sometimes enough in small projects, the third time we get bitten we start adding defensive checks so we get sensible exceptions: def method(self, alist, astring, afloat): if not isinstance(alist, list): raise TypeError('expected a list') if alist == []: raise ValueError('list must not be empty') # and so on... These are pre-conditions! We just don't call them that. And we can't disable them. They're not easy to extract from the source code and turn into specifications. And so much boilerplate! Let's invent syntax to make it more obvious what is going on: def method(self, alist, astring, afloat): requires: isinstance(alist, list) alist != [] isinstance(astring, str) number_of_vowels(astring) > 0 isinstance(afloat, float) not math.isnan(afloat) implementation: # code goes here Its easy to distinguish the precondition checks from the implementation, easy to ignore the checks if you don't care about them, and easy for an external tool to analyse. What's not to like about contracts? You're already using them, just in an ad hoc, ugly, informal way. And I think that is probably the crux of the matter. Most people are lazy, and don't like having to do things in a systematic manner. For years, programmers resisted writing functions, because unstructured code was easier. We still resist writing documentation, because "its obvious from the source code" is easier. We resist writing even loose specifications, because NOT writing them is easier. We resist writing tests unless the project demands it. We resist running a linter or static checker over our code ("it runs, that means there are no errors"). Until peer pressure and pain makes us do so, then we love them and could not imagine going back to the bad old days before static analysis and linters and unit tests. -- Steve
participants (3)
-
David Mertz -
Marko Ristin-Kaufmann -
Steven D'Aprano