Pre-conditions and post-conditions
Hi, I would be very interested to bring design-by-contract into python 3. I find design-by-contract particularly interesting and indispensable for larger projects and automatic generation of unit tests. I looked at some of the packages found on pypi and also we rolled our own solution (https://github.com/Parquery/icontract/). I also looked into https://www.python.org/dev/peps/pep-0316/. However, all the current solutions seem quite clunky to me. The decorators involve an unnecessary computational overhead and the implementation of icontract became quite tricky once we wanted to get the default values of the decorated function. Could somebody update me on the state of the discussion on this matter? I'm very grateful for any feedback on this!
You might also be interested in pep-563 <https://www.python.org/dev/peps/pep-0563/>. although it is not intended for design by contract, it can help (syntactically). Elazar On Wed, Aug 15, 2018 at 11:07 PM Marko Ristin-Kaufmann < marko.ristin@gmail.com> wrote:
Hi Marko Thank you for introducing yourself, and clearly stating your question. That helps us all. You asked:
Could somebody update me on the state of the discussion on this matter?
I think bring the existing PEP up to date would be a good starting point. Its content hasn't been changed since 2003 (except for PEP-wide admin changes. (Recall that Python 3.0 was released in 2008.) https://www.python.org/dev/peps/pep-0316/ https://github.com/python/peps/commits/master/pep-0316.txt In fact, revising the PEP might be enough to answer your question. What do you think, Marko? Experts: is there a process for revising old PEPs, such as this one? Or at least a precedent we could follow (or adapt)? -- Jonathan
On Thu, 16 Aug 2018 at 10:41, Jonathan Fine <jfine2358@gmail.com> wrote:
I'm not aware of a formal process, but I'd have thought the following steps would be a reasonable approach: 1. Review the PEP, and research the discussions that happened at the time, particularly of interest is why the PEP was deferred. 2. Consider what (if anything) has changed since the original deferral (which could simply be "time has moved on, people's views may have changed" but ideally would include a bit more in the way of concrete motivation). 3. Contact the original PEP author and ask if he is interested in reopening the discussion, collaborating on a revision, or handing the PEP over. 4. Start up a discussion here, pointing out the original PEP and summarising the previous debate and why you want to restart the discussion. If you're hoping to change the details of the original PEP, summarise your changes and why you feel they are an improvement over the original. To answer the OP's question more directly:
Could somebody update me on the state of the discussion on this matter?
As far as I am aware, there has been no discussion on this subject since the PEP 316 discussions which ended up in its deferral. Elazar mentioned PEP 563, and there *may* have been mention of design by contract uses in the discussions on that PEP, but you'd have to search the mailing list archives to confirm that one way or another. Hence the suggestions that if you want to restart discussion, reviving PEP 316 is likely the best approach. Paul
Hi Jonathan and Paul, Thank you very much for your suggestions! I will try to contact the author of the PEP. Let me clarify a bit a potential misunderstanding. Please mind that contracts are not tied to individual variables, but to expressions. Think of it as defining a lambda which takes as input all the arguments of the function (and a result variable in case of post-conditions) which always needs to evaluate to True. Cheers, Marko Le jeu. 16 août 2018 à 12:24, Paul Moore <p.f.moore@gmail.com> a écrit :
I think that annotations were suggested because you could write an expression there without getting evaluated. I've thought about this problem many times in the past (as a Python dev with a long history working in Eiffel too).... For me one of the crucial issue that is hard to translate into the python model is that the assertions (say, a function precondition) are not conceptually part of the function itself, but the interface of the class. The "natural" python ways of attaching these assertions somehow to the function object do not work when you also use inheritance, because when you override a method the new function object clobbers the previous one. I've experimented at some point on how to put them in classes (and doing metaclass or __getattribute__ tricks) but nothing convinced me). In general, the way that python puts method call and inheritance semantic in a specific layout of runtime objects (which in general is really clever) seems to be a bit alien to the DbC idea where the asbtraction/interface of the class is conceptually separate and has independent information wrt to the runtime objects. On 16 August 2018 at 18:49, Marko Ristin-Kaufmann <marko.ristin@gmail.com> wrote:
-- <https://www.machinalis.com/> Daniel Moisset Technical Leader A: 1 Fore St, EC2Y 9DT London <https://goo.gl/maps/pH9BBLgE8dG2> P: +44 7398 827139 <+44+7398+827139> M: dmoisset@machinalis.com <dmoisset@machinalis.com> | S: dmoisset <http://www.linkedin.com/company/456525> <http://www.twitter.com/machinalis> <http://www.facebook.com/machinalis> <https://www.instagram.com/machinalis.life/>
pycontracts may be worth a look. https://andreacensi.github.io/contracts/ - @contract decorator, annotations, docstrings IDK if pycontracts supports runtime parameter validations that involve more than one parameter. Inheritance does appear to be supported, as are numpy array dimension constraints. I can't recall whether the pycontracts expression language precedes MyPy compile-time annotations; both with one syntax really would be great. On Monday, August 20, 2018, Daniel Moisset <dmoisset@machinalis.com> wrote:
Hi, I had a look at the messages related to the PEP in question (PEP 316) in the archive. As far as I can tell, the main objection is that you can achieve contracts by implementing it with decorators. I think that these objections miss what actually Daniel Moisset wrote in his message: contracts are more than pre- and post-condition checks on a function. The inheritance of decorators does not imply just inheriting the pre- and post-conditions, but also relaxing and tightening them (represented by "require else" and "ensure then" in Eiffel). If this is to be used effectively in practice with little overhead then we would either need to introduce new syntax to the language or make the compiler improve the byte code on the fly. Is there any chance to introduce these constructs in the language or is it too small a feature for such a big change? Since we wanted to have contracts in Golang, we implemented a tool that synchronizes the documentation of a function with the function code ( https://github.com/Parquery/gocontracts). Maybe this is the easier path to follow in Python as well? @Wes Turner: thanks for pointing to pycontracts. I'm aware of the library. It implements only contracts based on a single property. We found that limiting and rolled out our own solution that suited us much better: https://github.com/Parquery/icontract/ I also found informative messages on contract breach to be particularly important for fast development and error inspection in the production. Cheers, Marko On 21 August 2018 at 04:44, Wes Turner <wes.turner@gmail.com> wrote:
On Tue, Aug 21, 2018 at 09:06:54AM +0200, Marko Ristin-Kaufmann wrote:
Is there any chance to introduce these constructs in the language or is it too small a feature for such a big change?
I don't think contracts is a small feature. I think it is a HUGE feature, but under-appreciated by most developers. (Probably due to unfamiliarity, and the difficulty in using it in a language with no syntactic support.) Whether it is practical to add it to Python, I don't know. I suspect that we would have to develop some sort of third-party solution first, even if it did not do everything contracts ought to do (or do them less efficiently) as a proof of concept. PyContracts is probably a good place to start. For those who missed Wes' email: https://andreacensi.github.io/contracts/ Cobra is another good place to look, as it demonstrates a nice syntax that reads like a cross between Python and Eiffel: http://cobra-language.com/trac/cobra/wiki/Contracts -- Steve
On Tue, 21 Aug 2018 at 11:27, Steven D'Aprano <steve@pearwood.info> wrote:
Agreed. And it sounds like there are a lot of subtleties to contracts that I certainly hadn't appreciated (I don't know if the same is true of others). For example, On Tue, 21 Aug 2018 at 08:08, Marko Ristin-Kaufmann <marko.ristin@gmail.com> wrote:
I think that these objections miss what actually Daniel Moisset wrote in his message: contracts are more than pre- and post-condition checks on a function. The inheritance of decorators does not imply just inheriting the pre- and post-conditions, but also relaxing and tightening them (represented by "require else" and "ensure then" in Eiffel). If this is to be used effectively in practice with little overhead then we would either need to introduce new syntax to the language or make the compiler improve the byte code on the fly.
I've no idea what "relaxing" and "tightening" of contracts involves, or how it would translate into Python. So I'd imagine that in order to put together a proposal for adding contracts to Python, you'd need to explain what contracts are, and how they work, to get past people's preconceptions that "they are just assertions". Otherwise, it's likely to be hard to persuade people of the benefits. Personally, I don't really think I can comment much further, precisely because it looks like I don't know enough about what contracts are and how they'd be used to contribute :-) Paul
Hi, @Paul Moore: thanks for pointing out that many people are not familiar with design-by-contract. I was not aware of that. Let me give you a very short introduction into contracts and what they are good for. I'll review some existing libraries and highlight what features we missed (and why we developed our own library). I will finally conclude with why all these solutions (including our own one) are not a replacement for a proper support of design-by-contract in the language. Design-by-Contract Design-by-contract was gradually introduced as a concept by Bertrand Meyer in the 1980ies to provide a formal and verifiable interface specification between the software components. Up to then the interfaces were usually defined formally in terms of abstract data types (think of records / structs and classes). He extended abstract data types by adding "contract" conditions that should hold at different points during the execution of a program. The contracts allow you to formally write down your expectations about the program (as opposed to writing it informally in documentation of a class or a function). Hence we can automatically test that they hold either statically or during the execution of the program. This gives us many-fold benefits: - contracts prevent code rot (since they can be checked by a compiler or a static analysis tool such as mypy), - allow us to have much more sophisticated automatic generation of unit tests and automatic formal proofs of a program, - make the documentation explicit, accurate and verifiable and - accelerate the development since errors are caught early. The contracts are categorized as follows depending on at which point in the program they are verified: - Preconditions are contracts that should hold before the execution of a function. The *caller* is responsible to fulfill the preconditions. - Postconditions are contracts that should hold after the execution of a function. The *callee* is responsible to fulfill the postconditions. - Invariants should hold throughout the execution of the program. There are two types of invariants: loop invariants and class invariants. - Loop invariants should hold before and after each iteration step of a loop. - Class invariants should hold throughout the life time of a class ( *i.e.* between the calls to the public methods). The class invariants are suspended during the construction of an instance and in private methods to avoid cycles. You can think of the constructor method and public methods being responsible to fulfill the class invariants. The concept of design-by-contract is not limited only to concrete classes, but can be also applied to class hierarchies. Preconditions, postconditions and invariants are inherited. They can be also modified in the following ways: - The child class needs to fulfill all the invariants of its antecedent classes and its own ones. - The preconditions of a function of a child class can "weaken" or "relax" the preconditions of the parent class. In other words, it needs to fulfill *either* the preconditions of the parent class *or *its own set of preconditions. This is reflected in Eiffel by using the keyword *require else.* - The postconditions of a a child class can "strengthen" or "tighten" the postconditions of the parent class. The function needs to fulfill all the postconditions of the parent class' function *and *its own set of postconditions. In Eiffel, this is designated with *ensure then* keyword. Invariants operate only on the values of instance properties. Preconditions operate on both function arguments and instance properties. Postconditions need yet one more instrument: they operate on function arguments, instance properties and the result of a function, but can access all these values both at their *old* state, before the function call, and their *new *state, after the function call. In Eiffel, you use the keyword *old* to indicate the value before the function call. Let me illustrate the concepts by adapting the examples from https://www.eiffel.org/doc/eiffel/ET-_Design_by_Contract_%28tm%29%2C_Asserti... and https://www.eiffel.org/doc/eiffel/ET-_Inheritance. I will paraphrase the example code in Python assuming that invariant, require, ensure and old were introduced as keywords. *class *Account: *def *__init__(self, balance: int) -> *None*: self.balance = balance self.deposits = [] *# type: List[int] **def *update(self, sum: int)->*None*: require: sum >= 0 self.balance += sum self.deposits.append(sum) ensure: len(self.deposits) = len(old self.deposits) + 1, *"one more deposit" *ensure: self.balance = old self.balance + sum, *"updated" *invariant: *not *self.deposits *or *self.balance == sum(self.deposits), *"consistent balance" *invariant: self.deposits *or *self.balance == 0, *"zero if no deposits" class *CheckingAccount(Account): *def *update(self, sum: int) -> *None*: require *else*: self.balance >= sum self.balance += sum self.deposits.append(sum) invariant: self.balance >= 0 For more examples and details, please see the before-mentioned two web pages on Eiffel. Current Implementations A series of open-sourced libraries have been developed to bring design-by-contract to Python. *PyContracts* (https://pypi.org/project/PyContracts/) introduced the preconditions and postconditions as special annotations: @contract def my_function(a : 'int,>0', b : 'list[N],N>0') -> 'list[N]': # Requires b to be a nonempty list, and the return # value to have the same length. ... new_contract('valid_name', lambda s: isinstance(s, str) and len(s)>0) @contract(names='dict(int: (valid_name, int))') def process_accounting(records): ... I personally find these annotations hard to read. The contracts seem mostly focused on type checks and on single arguments. Moreover, without support in IDEs and static analysis tools, annotations are susceptible to code rot if not verified by a linter (as far as I could find out, there are not linters that check pycontracts -- please correct me if I'm wrong). I also don't see how you could elegantly implement a check based on multiple arguments (*e.g., *sum >= balance in the CheckingAccount example). *Dpcontracts* (https://pypi.org/project/dpcontracts/) is a library based on decorators. It encapsulates the arguments of a function and its result as the arguments of the condition function:
It also supports invariants:
Contracts (https://pypi.org/project/contracts/), pyadbc ( https://pypi.org/project/pyadbc/) and pcd (https://pypi.org/project/pcd/) are similar in terms of features to dpcontracts, but seem not to be maintained any more. We found that all the presented methods were a bit impractical to use in everyday programming. Pycontracts forces the programmer to learn a new syntax which is not statically checked. While we liked the features provided by dpcontracts, we found that exception messages thrown at contract breach were uninformative. The programmer is forced to repeat every condition in text if s/he is to make any use of the error once it happens. This can lead to mismatch between the messages and code (similar how the comments tend to rot) and can make debugging and tracing bugs very hard. Moreover, it is tedious and repetitive to document the conditions twice which makes programmers sometimes reluctant to adopt it and apply it widely. We therefore developed *icontract* (https://pypi.org/project/icontract/). We decided to leave out class invariants for practical reasons (we rarely use classes in our applications and we needed conditions in production for which the invariants are often impractical), but invariants can be easily added if there is a demand. icontract is based on decorators and uses lambda functions for the conditions that match the argument names of the function. The argument *result* is reserved for the result of the function in postconditions. Here is the example usage:
Mind that there is no mandatory description in the contract yet the message is informative. We achieve that by re-executing the condition function and tracing the values by examining its abstract syntax tree. The re-computation can also deal with more complex expressions and outer scope:
We found informative messages to be a tremendous booster when debugging since you often immediately see what caused the contract breach and not only that it was broken. This often points you directly to the cause of the bug. Moreover, conditions are much more readable when not cluttered by redundant descriptions. This is important when you use them as part of the documentation that you inspect with an IDE (*e.g., *in PyCharm). By our subjective impression, other solutions resulted in hard-to-read documentation in PyCharm. While descriptions can also be added to the icontract conditions, we rarely find that necessary since most conditions are self-explanatory. Insufficiencies of the Current Libraries All the libraries described here were not trivial to implement and come with a substantial computational overhead. In one of our benchmarks, we found that having a precondition made a function run at least 6x slower (we traced the slow-down to an additional function invocation which is costly in Python). I don't think that it is possible to implement *old *keyword for most practical applications since the execution would be even slower. I found no library so far that supports inheritance, strengthening (ensure then) and weakening (*require else)* of the contracts out-of-the-box. My intuition tells me that it is possible to implement such a feature in a library, but the implementation will definitely be very complex. Apart from computational efficiency and complexity of implementation, I also see the variety of libraries as a problem for the adoption of design-by-contract. With multiple solutions and each team having their own preferences there is a dilemma which solution to choose. Without a wide adoption of a single solution we can not expect an emergence of tools such as automatic test generators built on top of the contracts which is where the actual tremendous benefits really await. A standard solution would allow us to have uniform, widely-adopted and efficient design-by-contracts in Python which no library by itself can achieve. Sketch of a Solution Instead of introducing new keywords, Python could introduce a built-in module based on decorators. The interpreters could be extended such as that they in-line the code directly into the function whenever the decorators of this built-in module is encountered. This would substantially reduce the computational overhead while it would allow us to avoid changes to language syntax. However, before we even start looking at a solution, I see it necessary that we first discuss more to which degree contracts should be introduced to Python and what the use cases would look like. Cheers, Marko
Total noob speaking here, but.... Those contracts are mostly important during development right ? Slowdown isn't that much of an issue during development. So you could make a debug mode that enforces the contracts, and a production mode that code users can use during production to stop the slowdown - in this case, your decorators can return their functions/classes unaltered. If they do end up with problems, you can always ask them to run the same inputs with debug mode on to see where it goes wrong. For the rest, i'm totally new to design by contract. I do get the draw of it, but im not sure if I'd ever really use it. I tend to solve my problems with a royal sprinkling of logger.debug(f"{val_i_need_to_check}").
On Mon, Aug 27, 2018 at 09:24:20AM +0100, Ivan Levkivskyi wrote:
TBH, I think one of the main points of design by contract is that contracts are verified statically.
No, that's not correct. Contracts may be verified statically if the compiler is able to do so, but they are considered runtime checks. Static checks are an optimization. For example, the Eiffel docs describe one possible contract as "the graph contains no cycle" and can contain function calls. https://www.eiffel.org/doc/eiffel/ET-_Design_by_Contract_%28tm%29%2C_Asserti... -- Steven
On Mon, 27 Aug 2018 at 11:39, Steven D'Aprano <steve@pearwood.info> wrote:
Considered by whom? By people who prefer `assert isinstance(x, int)` over `x: int`? :-) Contract in 99% of cases is just another word for type (maybe a very complex type like `DAG[T] <: Graph[T]`). Everything else, like `x >= y` is better expressed as an explicit assert with an assert message. But again this is rather IMO, than any kind of definition. There is only one use case I see now where a dedicated syntax would give a large readability gain: something like `self.x >= self.y`. But on the other hand I think such situations are too rare to justify any _new_ syntax. -- Ivan
Ivan and Steve wrote
TBH, I think one of the main points of design by contract is that contracts are verified statically.
No, that's not correct.
https://www.eiffel.org/doc/eiffel/ET-_Design_by_Contract_%28tm%29%2C_Asserti...
The page from Steve supplied (URL above) states
Based on the Eiffel docs, I find Ivan's opinion reasonable. He said it was "one of the main points". The goal is to detect errors immediately. Run-time assertion monitoring and static typing are two means towards that end. Our shared problem and goal is to have similar immediate detection of errors in Python (when the development process requires that degree of rigour). -- Jonathan
I didn't want to embarass Ivan any further by seemingly picking on his opinion about contracts being always statically checked, but when I asked off-list I got told to reword and post it here. So here it is. Sorry Ivan if this makes you feel I'm picking on you, that isn't my intention. On Mon, Aug 27, 2018 at 12:25:44PM +0100, Jonathan Fine wrote: [..]
Ivan said that static checking was a main point. Those Eiffel docs which you (Jonathon) quoted approvingly describe them as "run-time assertions". You describe them as "run-time assertions". I described them as "run-time assertions". So I'm having difficulty in understand what part of Ivan's opinion that they are compile-time static checks is "reasonable". If there's something I'm missing about Ivan's comment that you can see, I'd like to be enlightened. I don't see the relevance of the "two means towards that end" -- we have many means towards detecting bugs as early as possible: - correctness proofs - test-driven development - type checking - design by contract (and possibly more). If it was just a throw-away comment and I'm reading more into it than you intended, that's okay too, but I'd like to understand what you meant.
Well, yes, but what's that got to do with the question of whether contracts are checked statically or at runtime? -- Steve
This is about a difference of opinion regarding design by contract and static checking, that Steve D'Aprano has re-raised. Steve wrote that Ivan Levkivskyi's opinion was that:
contracts [are] always statically checked
This is what Ivan wrote:
TBH, I think one of the main points of design by contract is that contracts are verified statically.
There's no 'always' or 'all' here. I read it to mean 'sometimes' or 'some'. And also, that static verification is a good thing. My message of support for Ivan quoted the Eiffel docs.
https://www.eiffel.org/doc/eiffel/ET-_Design_by_Contract_%28tm%29%2C_Asserti...
I then wrote:
Based on the Eiffel docs, I find Ivan's opinion reasonable.
Steve wrote:
I'm sorry, but I'm just not seeing that. Either in what I quoted, or elsewhere in the page. True, my quote is from a section titled "Run-time assertion monitoring", but the context to me makes it clear that in Eiffel static typing IS NOT regarded as a run-time assertion. By the way, Steve wrote
when I asked off-list I got told to reword and post it here.
I don't think that's quite right. What I said off-list to Steve was
If you post [...] to python-ideas, I'd be happy to respond there.
It was my intention that it was up to Steve, whether or not to re-raise the issue. And now I'm doing my part of the bargain, by responding happily. I'm now happy to let this particular topic rest. -- Jonathan
On Wed, Aug 29, 2018 at 02:26:54PM +0100, Jonathan Fine wrote:
Fair enough, I should not have added "always" in my description. But you probably shouldn't have skipped the part I wrote earlier: "Contracts may be verified statically if the compiler is able to do so, but they are considered runtime checks. Static checks are an optimization." In context, you quoted me disagreeing with the "static" claim, but trimmed out my qualification that contracts may sometimes be statically verified when possible. [...]
Of course it isn't. By definition, static typing is done *statically*, at compile-time, not run-time. That has not been questioned and nobody has asserted that static typing is done at run-time. We're discussing whether *contracts* are checked at run-time. Because contracts are (in general) run-time assertions, they are a good fit for Python's execution model -- or at least they would be if we can find a good way to apply contracts to methods and classes. If they were static, they would be a bad fit and would probably need to be handled by a separate static checker, like MyPy does for static type checks. -- Steve
Jonathan Fine wrote:
I think you're misinterpreting the Eiffel docs here. It's saying that contracts *together* with static typing help to catch a lot of errors early in the development process. It's not saying that contracts are verified statically, or that all the errors thus caught are caught at compile time.
the context to me makes it clear that in Eiffel static typing IS NOT regarded as a run-time assertion.
That's true, but static typing and contracts are *different things* in Eiffel. Static types are checked at compile time, contracts are checked at run time. -- Greg
Hi, I think we got entangled in a discussion about whether design-by-contract is useful or not. IMO, the personal experience ("I never used/needed this feature") is quite an inappropriate rule whether something needs to be introduced into the language or not. There seems to be evidence that design-by-contract is useful. Let me cite Bertrand Meyer from his article "Why not program right?" that I already mentioned before:
6. Bertrand Meyer, Object-Oriented Software Construction, 2nd edition,
Is there any logical or empirical objection that the design-by-contract is not useful and hence does not merit to be introduced into the core language? There is little point in writing a PEP and fleshing out the details if the community will reject it on grounds that design-by-contract by itself is meaningless. So I'd suggest we clarify this first before we move on. @David Mertz:
Could you please elaborate a bit? I don't see how the annotations would make the contracts invoked on inheritance. Consider this simple case: class A: @icontract.pre(lambda x: x > 0) def some_method(self, x: int)->None: pass class B(A): # Precondition should be inherited here. def some_method(self, x: int) -> None: pass You would still need to somehow decorate manually the overridden methods even though you would not specify any new contracts, right? Is there a mechanism in Python that I am not aware of that would allow us to accomplish that? On Wed, 29 Aug 2018 at 22:41, Greg Ewing <greg.ewing@canterbury.ac.nz> wrote:
On Wed, Aug 29, 2018 at 3:07 PM Marko Ristin-Kaufmann < marko.ristin@gmail.com> wrote:
A metaclass does this pretty easily. I have a thing I wrote years ago called MIS (Multi-Inheritance Safety) that is used to ensure you don't do anything stupid in our physics-modeling database. The database is a collection of ~200 Python classes/mixins all deriving madly from each other to get various behaviors (Has mass? You'll need this. Has moments? You'll need this. Has physical extent, i.e., can be seen? You'll need these...). Anyhow, the basemost class has the metaclass, which traverses all the methods in the subclasses and makes sure you don't have methods with the same name in two distinct superclass trees and such things. It also forces you to be explicit when you overload a method from a base class (among other things, but this one is easy to illustrate). class MIS(type): def __init__(cls, name, bases, namespace): mro = cls.mro()[1:-1] # All the stuff between new class and 'object' for method_name, method in namespace.items(): if isinstance(method, executable_types): if not getattr(method, '_its_ok', False): # Make sure it's not in any baser class. # Could easily check for decorated pre/post properties and copy them... def override(func): # Mark the function as a valid override... func._its_ok = True return func class Base(metaclass=MIS): def something(self): pass class Derived(Base): @override # Signal that this is ok, otherwise we get an error. def something(self): ...
The technique Eric suggests is probably better than what I had in mind. But I was thinking you could have an "inherit" decorator for methods (or for a class as a whole). It's easy enough for a decorator to attach a `.__contracts__` attribute to either the class or the individual methods. Then the decorator(s) in the child can simply look through the `.__mro__` to find any such parent contracts. E.g.: class B(A): @inherit_invariants def some_method(self, x: int) -> None: pass @precondition(lambda x: x=42, inherit_parent=True) def other_method(self, x: int) -> float: return 42/5 I'm not writing a library to do this, so you can tweak the API to be different than my example. But this is already well possible. On the broader idea: Is there any logical or empirical objection that the design-by-contract is not useful and hence does not merit to be introduced into the core language? There is little point in writing a PEP and fleshing out the details if the community will reject it on grounds that design-by-contract by itself is meaningless. Lots of people have explained this relative to lots of ideas, probably mostly better than I will. Adding a new feature, even if it is *technically* backwards compatible, has HUGE costs. All the documentation—the books, articles, blog posts, videos, webinars, tutorials, etc.—about Python has to be updated. We get a divide between "code that will work in Python 3.9" versus what will run in 3.7. The cognitive burden of learning Python is increased for everyone in the world (millions of people) because even if they do not use a feature they will encounter code that does. There is another section of code in the implementation(s) of Python that can potentially have bugs and needs to be maintained by someone. Obviously, design-by-contract is not *meaningless*! It's a specific feature that is relatively well defined as a concept (and precisely defined in regard to Eiffel specifically; but we might not implement those *exact* semantics). It's also a feature that no languages in particularly widespread use have decided to implement at the language level. I've chatted with Meyer; he's definitely very smart and definitely strongly opinionated, but I also think he's wrong about the overall importance of this feature versus lots of others. In my mind, this feature doesn't come close to meeting the burden of those high costs listed above (and others I did not mention). But I don't have any say in what the core developers will do, beyond in that they might be influenced by my opinion here. Yours, David... On Wed, Aug 29, 2018 at 6:41 PM Eric Fahlgren <ericfahlgren@gmail.com> wrote:
-- Keeping medicines from the bloodstreams of the sick; food from the bellies of the hungry; books from the hands of the uneducated; technology from the underdeveloped; and putting advocates of freedom in prisons. Intellectual property is to the 21st century what the slave trade was to the 16th.
On 8/29/2018 6:39 PM, Eric Fahlgren wrote:
I think a metaclass is a non-starter. If one were used, it would preclude using contracts in any case where a metaclass were already used, or where one was needed in the future. I'm sure people will disagree with me on this. But, I think a more productive line of thinking is: what could be added to the language that would let contracts be implementable, and could also be used for other things, too? Sort of like how PEP 487 adds customizability that has wide applicability. Eric
On Wed, 29 Aug 2018 at 23:08, Marko Ristin-Kaufmann <marko.ristin@gmail.com> wrote:
Hi, I think we got entangled in a discussion about whether design-by-contract is useful or not. IMO, the personal experience ("I never used/needed this feature") is quite an inappropriate rule whether something needs to be introduced into the language or not.
It's not a key factor, but it is indicative, in the sense that if no-one has ever needed the feature *in Python*, then it's possibly not a good fit for how the language is used in real life. That doesn't exclude the possibility of a new feature offering a new and not previously considered technique, but if that's the assertion, then it's up to the individual proposing the new feature to persuade the community that there's a real and significant benefit.
Is there any logical or empirical objection that the design-by-contract is not useful and hence does not merit to be introduced into the core language? There is little point in writing a PEP and fleshing out the details if the community will reject it on grounds that design-by-contract by itself is meaningless. So I'd suggest we clarify this first before we move on.
"It could be useful" is nowhere near a strong enough reason for changing the language. Even "it is demonstrably useful in other languages" would have a hard time. What you need to demonstrate is that it would be useful *in Python code*. A good way of doing that is by reviewing a significant body of real-life Python code (the standard library is a common choice, but a big project like Django or SQLAlchemy would also be reasonable) and demonstrating how the code could be *improved* by using the proposed new feature. Here, the key is that the change has to be an improvement - readability is one (somewhat difficult to assess objectively) criterion, as is ease of identifying bugs, or efficiency (less need to repeat complex expressions, for example). Conversely, any proposed new feature needs to address how it impacts people who *don't* want to use it. That can be as simple as backward compatibility issues such as adding new keywords (people using that keyword as a variable name will have to change their code) but can also address any general runtime impact in the interpreter (bookkeeping and management of new data structures for example, or changes to class implementation details). and how people who don't use the new feature will be impacted if they encounter the feature in code written by others, or in code reviews. Also, there's the question of teachability. A proposed new feature must address how it will be explained to new and existing Python users. And given the confusion we're seeing in this thread ("aren't these just runtime assertions?") that's something that contracts will definitely have to address. I'm not saying that the proposal needs to offer a full tutorial on design by contract, but at a minimum, it'll have to cover why contracts aren't just runtime assertions, and how the differences can be made clear in the documentation and by trainers. I hope that helps explain what you'll need to do if you want to take this proposal forward, and why you're getting pushback in areas that maybe seem incidental to you. Personally, I'm interested in the feature, but I'm not sure I'm interested enough to want it in Python. My main questions are 1. How exactly do these differ from simple assertions? I don't understand the hints about "relaxing" and "strengthening" contracts in subclasses, in particular I've no idea how I'd express that in actual syntax. 2. Are we talking here about adding checks that would run in production code? Wouldn't that slow code down? How do I deal with the conflict between wanting tighter checks but not wanting to pay the cost at runtime of checking things that should never go wrong (contracts, as I understand it, are for protecting against code bugs, that's why there's no facility for trapping them and producing "user friendly" error messages)? I use assertions very sparingly in production code for precisely that reason - why would I be more willing to use contracts? Paul
On Mon, Aug 27, 2018 at 12:12:22PM +0100, Ivan Levkivskyi wrote:
No, considered by Bertrand Meyer, the inventor of Design By Contract and the programming language Eiffel. Contracts are not limited to the things which static type-checkers are capable of testing, but can and do involve checks which are impossible or impractical to test at compile-time, like "today is Tuesday" or "the account balance is greater than the amount you are trying to withdraw".
Contracts ARE assertions. They are assertions about the input a method expects, not merely the type but the value, what result it intends to return, and the invariants of the class after the method is called. Like assertions, they are called at runtime, and can be disabled. Using contracts to enforce type-correctness wouild be silly in Eiffel, because Eiffel already has a powerful static type system. Sybtax-wise, if you're interested in what is possible in a Python-like language, you could do worse than check out Cobra: http://cobra-language.com/trac/cobra/wiki/Contracts -- Steve
On Mon, Aug 27, 2018 at 10:50 PM, Steven D'Aprano <steve@pearwood.info> wrote:
Sometimes "type" doesn't mean the same thing to the language and to the human. Suppose you're trying to create a Python script that replicates a C program; you might want to declare that a variable is not of type "integer" but type "32-bit unsigned integer", with wrap-around. Or, wording it another way: "integer modulo 2**32". Is that an assertion of type, or of type and value? As a precondition to a function, requiring that a parameter be an integer no less than zero and no greater than 4294967295 is, in a sense, checking its type and its value; but it's kinda just asserting its type. AIUI, that's what Ivan meant by "complex type" - something that determines the domain of valid values as well as the concrete type. ChrisA
On Mon, Aug 27, 2018 at 11:00:22PM +1000, Chris Angelico wrote:
It is making an assertion about the value of an instance of type "int". Its not a separate type requiring an explicit coercion or cast. Its just a non-negative int less than 2**32. If the compiler supports the concept of a 32-bit unsigned integer type, then of course we can change our implementation to use that type instead of our regular ints. But we can't expect to pass such a 32-bit unsigned integer to a function which expects a regular int unless they are duck-type compatible, or the compiler performs automatic coercions. (So-called "weak typing"). A better example is the one I gave earlier, of a graph with no cycles. There is a deep fundamental difference between a *statically checked* DAG with no cycles (a graph which can never contain a cycle because the compiler won't let you create one) and a *dynamically checked* DAG that merely has no cycles *now* (it may have had cycles earlier, and it might have cycles later, but right now it has none). These are very different semantics, and Eiffel's contracts support the second kind: runtime value checks. -- Steve
Runtime checks: data validation & code validation Compile-time checks: code validation What sort of data validation is appropriate for assert statements or contacts that may be skipped due to trading performance for more risk ('optimized out')? Checking the value of a Number? Checking that a large graph has no cycles? Checking that a database table exists and has the appropriate relations and constraints? assert statements are skipped at runtime with -O and -OO whether or not they're in [reorderable] aspects applied with decorators, at the beginning or end of a function, or in methods named something like setUp and tearDown. https://docs.python.org/3/using/cmdline.html#envvar-PYTHONOPTIMIZE https://docs.python.org/3/using/cmdline.html#cmdoption-o
On Monday, August 27, 2018, Steven D'Aprano <steve@pearwood.info> wrote:
On Mon, Aug 27, 2018 at 11:01:21AM -0400, Wes Turner wrote:
That depends on what you mean by "data validation". Testing for bad input, or testing for predictable error states such as I/O errors, missing files, permission errors, server down etc is not appropriate for assertions (which includes contracts). The rule I use is that assertions are for: (1) testing your program state, which is under your control; and (2) communicating the intention of your program as executable code rather than comments. The ultimate aim of contracts and assertions is to eventually disable them when the program is bug-free. The Eiffel docs say: It should be clear from the preceding discussion that contracts are not a mechanism to test for special conditions, for example erroneous user input. For that purpose, the usual control structures [...] are available [...] An assertion is instead a correctness condition governing the relationship between two software modules (not a software module and a human, or a software module and an external device). ... Bluntly: Rule -- Assertion Violation: A run-time assertion violation is the manifestation of a bug. https://www.eiffel.org/doc/eiffel/ET-_Design_by_Contract_%28tm%29%2C_Asserti... For detecting *external* error states (anything to do with data that comes from outside your program, like user input) you can never let your guard down and never disable the test, because servers can always go down, users can always give you bad data, files can always be corrupt. It is unsafe to disable these tests and so these should not be assertions. For a library function intended to be called by third-parties, the function arguments aren't under the control of the library author so should not be tested with assertions. But for an application where the author controls those function arguments, they are under the author's control and may be assertions or contracts. Design By Contract is partly a methodology and partly a set of syntax. Its a way of thinking about the design of your program. In practice, you don't have to buy 100% into DBC to get some benefit for it. A study done a few years back looked at 21 large projects in Eiffel, JML (Java) and Code Contracts for C# and found that 33% of the classes used contracts. http://se.ethz.ch/~meyer/publications/methodology/contract_analysis.pdf Like unit testing, you don't need 100% coverage to get benefit. 10% is better than nothing, and 20% is better than 10%. I wrote more about assertions here: https://import-that.dreamwidth.org/676.html -- Steve
Thanks for the explanation. This may be a bit OT, but is there a good way to do runtime assertions (in particular for data validation) that's as easy as assert? - self.assertEqual (unittest.TestCase.assertEqual) is hardly usable in other class instances, - pytest can be used at runtime but has somewhat nontrivial overhead - I always end up writing e.g. _assert() and _assertEqual() that throw AssertionErrors and helpful error messages just like unittest. How do contracts differ from checking interfaces with e.g. zope.interface? https://zopeinterface.readthedocs.io/en/latest/verify.html I'll read up a bit more on design by contract. I seem to have conceptualized dbc as a composition approach with typed interfaces and type and value assertions, but that's not it? If the parameter value assertions in pycontracts aren't really contracts, and the mypy compile-time parameter and return type checks are not really contracts, and zope.interface verifications aren't really contracts; what does that leave in terms of abstract data types, preconditions, postconditions, and invariants? https://en.wikipedia.org/wiki/Design_by_contract On Monday, August 27, 2018, Steven D'Aprano <steve@pearwood.info > wrote:
Hi, To clarify the benefits of the contracts, let me give you an example from our code base: @icontract.pre(lambda x: x >= 0) @icontract.pre(lambda y: y >= 0) @icontract.pre(lambda width: width >= 0) @icontract.pre(lambda height: height >= 0) @icontract.pre(lambda x, width, img: x + width <= pqry.opencv.width_of(img)) @icontract.pre(lambda y, height, img: y + height <= pqry.opencv.height_of(img)) @icontract.post(lambda self: (self.x, self.y) in self) @icontract.post(lambda self: (self.x + self.width - 1, self.y + self.height - 1) in self) @icontract.post(lambda self: (self.x + self.width, self.y + self.height) not in self) def __init__(self, img: np.ndarray, x: int, y: int, width: int, height: int) -> None: self.img = img[y:y + height, x:x + width].copy() self.x = x self.y = y self.width = width self.height = height def __contains__(self, xy: Tuple[int, int]) -> bool: x, y = xy return self.x <= x < self.x + self.width and \ self.y <= y < self.y + self.height We use mypy and type annotations for typing, and we don't need contracts for that. In this particular case, contracts are very handy to formulate how we deal with pixels. If you carefully look at the contracts, you will see: * pixels are indexed starting from 0 (as oppose to starting from 1) * region-of-interest needs to fit within an image. It can not be outside of its boundaries * pixels within the region-of-interest are in [x, x + width), [y, y + height) (where "[" means inclusive and ")" exclusive) *Why not types?* These constraints are either impossible or very hard to write as types. Moreover, I doubt that you could give such types meaningful names, and I expect the readability to suffer immensely. I suppose any constraints involving more than a couple of variables is hard to write as a type. *Benefits.* You could write all of this in human-readable informal docstring of a class, but then it would never be verified. This way we are sure that contracts are always there. Whenever we test, we can be sure that all the contracted properties hold. Mind that we do not test that they hold for that single test case -- we automatically test that they hold for all the test cases. This is, in my opinion, far superior to ambiguous human documentation or print statements (which are deleted after the debugging). I suppose *any* larger code base built by a team of more than one developer needs this kind of rigor. The contracts are not meant to be used only in high-risk applications -- they are meant to improve any code base. *Problem without standard support. *The current libraries (dpcontracts, icontract) can deal with pre and postconditions and class invariants of a concrete class. However, the current libraries break as soon as you have inheritance. There is no way in python to inherit the function decorators and to modify them. If we are to inherit from the above-mentioned class ROI, we need to make sure that the invariants of the parent class hold (*e.g., *what is contained in a ROI) as well as invariants of the child class. We might want to weaken the requirements (*e.g., *a ROI that can deal with pixels outside of an image) such that x and y can be an arbitrary numbers, not restricted to 0 <= x < img.width and 0 <= y < img.height respectively. Additionally, without standard approach, we do not know how to deal with contracts when we have a third-party tool that would like to use them (*e.g., *for automatic generation of unit tests, static testing or visualization in IDE or in documentation). @Wes Turner <wes.turner@gmail.com>: If I understood you correctly, you are looking for a library that gives you verbose messages when a contract is breached. Please have a look at icontract library: https://github.com/Parquery/icontract We added informative messages particularly because we wanted to have verbose output when something goes wrong.This was helpful not only during the development, but also in production since failure cases were hard to reproduce and anticipate in the first place (otherwise, they wouldn't have made it into the production). Here is an example with an error message:
On Tue, 28 Aug 2018 at 03:19, Wes Turner <wes.turner@gmail.com> wrote:
I now fully understand your use case. IIUC, what would make this easier is a syntax for defining preconditions, post conditions, and invariants that supports: * inheritance, composition (mixins) This isn't possible with decorators because there's no reference to the post-decorated function/method. A conventional method naming scheme and composition like _pre_check_[004]_height() would be unreasonably verbose; though it would then be possible to 'relax' specific contracts by name. * A better flow? Decorators subvert the normal top-down source layout (and so require up-down eye movements and working memory to remember what's going on) * Test case generation Generate tests and fixtures to verify the preconditions, postconditions, and invariants. * An Interpretor/compilation flag to skip the contracts/tests/assertions PYTHONOPTIMIZE (-O, -OO), __debug__ Isn't there now an -X <...> interpretor flag in CPython? On Tuesday, August 28, 2018, Marko Ristin-Kaufmann <marko.ristin@gmail.com> wrote:
I admit that I don't care that much about pre- and post-conditions. Like a lot of people I've written a you library to handle some cases, but was never motivates to make it better or use ones folks have refined. I definitely oppose dedicated syntax. Nonetheless, I'll say that it's really not hard to get a decorator to cooperate with inheritance if you want that. If you decorate a parent class you can simply attach the collection of invariants to the class as notations, along with whatever actual guards the decorator enforces. An inheriting child (also decorated) can just look up those notations and apply them to the child. I.e. the first thing a decorator can do is lookup the invariants attached to the parent (if any) and apply them to the child (if this behavior is enabled). Easy peasy. On Tue, Aug 28, 2018, 12:35 PM Wes Turner <wes.turner@gmail.com> wrote:
On Tue, Aug 28, 2018 at 07:46:02AM +0200, Marko Ristin-Kaufmann wrote:
Thanks for your example Marko. I think that is quite close to the ugliest piece of Python code I've ever seen, and I don't mean that as a criticism of you for writing it or the icontract library's design. We write code that our language allows us to write, not the code we want to write. Python has not been designed for contracts, and it shows. I think this demonstrates a truth about programming languages: Aethestics matter. Syntax matters. Some features, no matter how useful and beneficial, have no hope at all of getting widespread use if the syntax is awkward or the result ugly. I think that contracts will be doomed to be a minor niche used only by really keen proponents of the style so long as the best way to write a contract is to use a sequence of decorator calls with lambdas. I'm going to re-write that in a pseudo-Eiffel like syntax: def __init__(self, img: np.ndarray, x: int, y: int, width: int, height: int) -> None: require: x >= 0 y >= 0 width >= 0 height >= 0 x + width <= pqry.opencv.width_of(img) y + height <= pqry.opencv.height_of(img) # possibly this should be an invariant, not a post-condition? ensure: (self.x, self.y) in self (self.x + self.width - 1, self.y + self.height - 1) in self (self.x + self.width, self.y + self.height) not in self # body of __init__ goes here... This makes a huge difference to readability: - block structure to give visual shape to the code; - related code stays together (all the pre-conditions naturally go into the require block, post-conditions in the ensure block, and no chance of interleaving them @pre(...) @post(...) @pre(...) - no extraneous lambdas (they are pure noise); - no long sequence of decorators (more than one of each is pure noise); - possibility to write assertions which take more than one statement. -- Steve
How are conditions relaxed/overridden in Eiffel without a named reference? That looks much more readable. Expressions within such blocks are implicitly assertTrue(s). What sort of test generation from said nameless expressions would be most helpful? On Tuesday, August 28, 2018, Steven D'Aprano <steve@pearwood.info> wrote:
Wes Turner wrote:
I'm going to re-write that in a pseudo-Eiffel like syntax:
Maybe some magic could be done to make this work: def __init__(self, img: np.ndarray, x: int, y: int, width: int, height: int) -> None: def __require__(): x >= 0 y >= 0 width >= 0 height >= 0 x + width <= pqry.opencv.width_of(img) y + height <= pqry.opencv.height_of(img) def __ensure__(): (self.x, self.y) in self (self.x + self.width - 1, self.y + self.height - 1) in self (self.x + self.width, self.y + self.height) not in self # body of __init__ goes here... -- Greg
I've never used contracts, so excuse me if I didn't get how they would work, and what they should do. The last example is about pre-post conditions in a class constructor. But I imagine this is not the only case where one would want to define contracts, like probably: within methods that return something and to functions (and in both cases, we'd want to contractually state some of the return's specificities). Is the following function something someone used to work with contracts would write? def calculate(first: int, second: int) -> float: def __require__(): first > second second > 0 # or first > second > 0 ? def __ensure__(ret): # we automatically pass the return of the function to this one ret > 1 return first / second If so, having a reference to the function's output would probably be needed, as in the example above. Also, wouldn't someone who use contracts want the type hints he provided to be ensured without having to add requirements like `type(first) is int` or something? - Brice Le 29/08/2018 à 07:52, Greg Ewing a écrit :
On Wed, Aug 29, 2018 at 05:52:46PM +1200, Greg Ewing wrote:
The problem with this idea is that methods and functions are not declarations, but executable code. This __require__ function doesn't exist except while the __init__ method is running. So it can't be called before the __init__, it can't be called *automatically* (you need to call it yourself, from inside the __init__), and it can't be inherited. Of course with sufficient compiler magic of course the compiler could special case these methods and do whatever we want, but that seems like it would be just as much work but much uglier than using dedicated syntax. -- Steve
Op wo 29 aug. 2018 om 03:59 schreef Steven D'Aprano <steve@pearwood.info>:
What, really ? Well, it clearly shows you teach python and don't look much at code written by people who taught themselves. I taught myself, and the first .py file I created was over a 1000 lines, and contained the GUI in a 4-deep nested global dictionary, since I'd never seen a style guide at that point. (I got better)
On Wed, Aug 29, 2018 at 09:23:02AM +0200, Jacco van Dorp wrote:
[snip long sequence of @decorator(lambda) calls]
I said *close* :-)
Well, it clearly shows you teach python and don't look much at code written by people who taught themselves.
I didn't mean to compare it to code written by beginners. I meant professional quality. And I didn't mean it was *bad* code. Python is a remarkable elegant and pretty language, but there are some things that aren't a good fit to the existing syntax. Contracts are one. We can't easily write code in a declarative style like Prolog: sibling(X, Y) :- parent_child(Z, X), parent_child(Z, Y) (X is a sibling of Y when there exists some Z who is a parent of X and the same Z is the parent of Y); we have to re-write it in a procedural style. Some programming styles aren't a natural fit to a given syntax.
So far, nothing you describe is *necessarily* ugly or bad code. The std lib contains at least one file well over 1000 lines, and while it is complex code, its not ugly code by any means. And I couldn't judge the elegance of the dict unless I saw it and the alternatives :-) -- Steve
On Mon, Aug 27, 2018 at 09:04:21AM +0200, Jacco van Dorp wrote:
Total noob speaking here, but....
Those contracts are mostly important during development right ?
That assumes that production code is free of bugs. Usual practice in the Eiffel world is, I believe, to disable post- condition checking and other assertions in production code, but leave pre-condition checks in place. Note: for those unfamiliar with Design By Contract, pre- and post- condition checks and assertions are NOT used to detect predictable error states, such as bad user input or IO errors. They are only used for testing program correctness. A failed contract or assertion can only mean a bug in the program. In a totally bug-free program contracts can safely be disabled.
Slowdown isn't that much of an issue during development.
For many cases, it isn't an issue during production either.
In Eiffel, contracts can be disabled or enable globally, or on a class-by-class basis.
The differences between design by contract and sprinkling messages in the log include: - the compiler (or interpreter) never forgets to read the logs looking for failures; - when a failure occurs, the program halts, so you know its broken, rather than silently continuing and perhaps doing something unexpected; - you don't need to read the logs and try to remember why it is that you're printing the value of ``foo`` and whether its a good thing or a bad thing that it has the value you're seeing. Here's an introduction to design by contract: https://www.eiffel.org/doc/eiffel/ET-_Design_by_Contract_%28tm%29%2C_Asserti... -- Steve
On Thu, Aug 16, 2018 at 4:19 PM Elazar <elazarg@gmail.com> wrote:
You might also be interested in pep-563. although it is not intended for design by contract, it can help (syntactically).
FYI, PEP 563 doesn't help it. Read this section: https://www.python.org/dev/peps/pep-0563/#non-typing-usage-of-annotations "With this in mind, uses for annotations incompatible with the aforementioned PEPs should be considered deprecated." -- INADA Naoki <songofacandy@gmail.com>
Isn't the purpose of "assert" to be able to do design by contract ? assert test, "error message is the test fail" I mean, you just write your test, dev get a feedback on problems, and prod can remove all assert using -o. What more do you need ? Le 15/08/2018 à 23:06, Marko Ristin-Kaufmann a écrit :
Michel Desmoulin wrote:
Good question. My opinion is that assert statements are good. I like them. But wait, more is possible. Here are some ideas. 1. Checking the return value (or exception). This is a post-condition. 2. Checking return value, knowing the input values. This is a more sophisticated post-condition. 3. Adding checks around an untrusted function - possibly third party, possibly written in C. 4. Selective turning on and off of checking. The last two, selective checks around untrusted functions, I find particularly interesting. Suppose you have a solid, trusted, well-tested and reliable system. And you add, or change, a function called wibble(). In this situation, errors are most likely to be in wibble(), or in the interface to wibble(). So which checks are most valuable? I suggest the answer is 1. Checks internal to wibble. 2. Pre-conditions and post-conditions for wibble 3. Pre-conditions for any function called by wibble. Suppose wibble calls wobble. We should certainly have the system check wobble's preconditions, in this situation. But we don't need wobble to run checks all the time. Only when the immediate caller is wibble. I think assertions and design-by-contract point in similar directions. But design-by-contract takes you further, and is I suspect more valuable when the system being built is large. Thank you, Michel, for your good question. -- Jonathan
Hi, I implemented the inheritance via meta classes and function and class attributes for pre/postconditions and invariants, respectively. Unless I missed something, this is as far as we can go without the proper language support with a library based on decorators: https://github.com/Parquery/icontract (version 1.5.0) Note that it is actually a complete implementation of design-by-contract that supports both weakening of the preconditions and strengthening of the postconditions and invariants. Could you please have a look and let me know what you think about the current implementation? Once we are sure that there is nothing obvious missing, I'd like to move forward and discuss whether we could add this library (or rewrite it) into the standard Python libraries and what needs to be all fixed till to make it that far. Cheers, Marko On Sat, 8 Sep 2018 at 21:34, Jonathan Fine <jfine2358@gmail.com> wrote:
Hi, A brief follow-up (latest version 1.5.3): I removed the dependency on meta package so that now all comprehensions and generator expressions work. I still had to depend on asttokens in order to get the source code of the condition function. Is there maybe an alternative solution which uses only standard libraries? Any thoughts or feedback on the icontract library in general? Cheers, Marko On Mon, 10 Sep 2018 at 09:29, Marko Ristin-Kaufmann <marko.ristin@gmail.com> wrote:
Hi, Let me make a couple of practical examples from the work-in-progress ( https://github.com/Parquery/pypackagery, branch mristin/initial-version) to illustrate again the usefulness of the contracts and why they are, in my opinion, superior to assertions and unit tests. What follows is a list of function signatures decorated with contracts from pypackagery library preceded by a human-readable description of the contracts. The invariants tell us what format to expect from the related string properties. @icontract.inv(lambda self: self.name.strip() == self.name) @icontract.inv(lambda self: self.line.endswith("\n")) class Requirement: """Represent a requirement in requirements.txt.""" def __init__(self, name: str, line: str) -> None: """ Initialize. :param name: package name :param line: line in the requirements.txt file """ ... The postcondition tells us that the resulting map keys the values on their name property. @icontract.post(lambda result: all(val.name == key for key, val in result.items())) def parse_requirements(text: str, filename: str = '<unknown>') -> Mapping[str, Requirement]: """ Parse requirements file and return package name -> package requirement as in requirements.txt :param text: content of the ``requirements.txt`` :param filename: where we got the ``requirements.txt`` from (URL or path) :return: name of the requirement (*i.e.* pip package) -> parsed requirement """ ... The postcondition ensures that the resulting list contains only unique elements. Mind that if you returned a set, the order would have been lost. @icontract.post(lambda result: len(result) == len(set(result)), enabled=icontract.SLOW) def missing_requirements(module_to_requirement: Mapping[str, str], requirements: Mapping[str, Requirement]) -> List[str]: """ List requirements from module_to_requirement missing in the ``requirements``. :param module_to_requirement: parsed ``module_to_requiremnt.tsv`` :param requirements: parsed ``requirements.txt`` :return: list of requirement names """ ... Here is a bit more complex example. - The precondition A requires that all the supplied relative paths (rel_paths) are indeed relative (as opposed to absolute). - The postcondition B ensures that the initial set of paths (given in rel_paths) is included in the results. - The postcondition C ensures that the requirements in the results are the subset of the given requirements. - The precondition D requires that there are no missing requirements (*i.e. *that each requirement in the given module_to_requirement is also defined in the given requirements). @icontract.pre(lambda rel_paths: all(rel_pth.root == "" for rel_pth in rel_paths)) # A @icontract.post( lambda rel_paths, result: all(pth in result.rel_paths for pth in rel_paths), enabled=icontract.SLOW, description="Initial relative paths included") # B @icontract.post( lambda requirements, result: all(req.name in requirements for req in result.requirements), enabled=icontract.SLOW) # C @icontract.pre( lambda requirements, module_to_requirement: missing_requirements(module_to_requirement, requirements) == [], enabled=icontract.SLOW) # D def collect_dependency_graph(root_dir: pathlib.Path, rel_paths: List[pathlib.Path], requirements: Mapping[str, Requirement], module_to_requirement: Mapping[str, str]) -> Package: """ Collect the dependency graph of the initial set of python files from the code base. :param root_dir: root directory of the codebase such as "/home/marko/workspace/pqry/production/src/py" :param rel_paths: initial set of python files that we want to package. These paths are relative to root_dir. :param requirements: requirements of the whole code base, mapped by package name :param module_to_requirement: module to requirement correspondence of the whole code base :return: resolved depedendency graph including the given initial relative paths, """ I hope these examples convince you (at least a little bit :-)) that contracts are easier and clearer to write than asserts. As noted before in this thread, you can have the same *behavior* with asserts as long as you don't need to inherit the contracts. But the contract decorators make it very explicit what conditions should hold *without* having to look into the implementation. Moreover, it is very hard to ensure the postconditions with asserts as soon as you have a complex control flow since you would need to duplicate the assert at every return statement. (You could implement a context manager that ensures the postconditions, but a context manager is not more readable than decorators and you have to duplicate them as documentation in the docstring). In my view, contracts are also superior to many kinds of tests. As the contracts are *always* enforced, they also enforce the correctness throughout the program execution whereas the unit tests and doctests only cover a list of selected cases. Furthermore, writing the contracts in these examples as doctests or unit tests would escape the attention of most less experienced programmers which are not used to read unit tests as documentation. Finally, these unit tests would be much harder to read than the decorators (*e.g.*, the unit test would supply invalid arguments and then check for ValueError which is already a much more convoluted piece of code than the preconditions and postconditions as decorators. Such testing code also lives in a file separate from the original implementation making it much harder to locate and maintain). Mind that the contracts *do not* *replace* the unit tests or the doctests. The contracts make merely tests obsolete that test that the function or class actually observes the contracts. Design-by-contract helps you skip those tests and focus on the more complex ones that test the behavior. Another positive effect of the contracts is that they make your tests deeper: if you specified the contracts throughout the code base, a test of a function that calls other functions in its implementation will also make sure that all the contracts of that other functions hold. This can be difficult to implement with standard unit test frameworks. Another aspect of the design-by-contract, which is IMO ignored quite often, is the educational one. Contracts force the programmer to actually sit down and think *formally* about the inputs and the outputs (hopefully?) *before* she starts to implement a function. Since many schools use Python to teach programming (especially at high school level), I imagine writing contracts of a function to be a very good exercise in formal thinking for the students. Please let me know what points *do not *convince you that Python needs contracts (in whatever form -- be it as a standard library, be it as a language construct, be it as a widely adopted and collectively maintained third-party library). I would be very glad to address these points in my next message(s). Cheers, Marko
I'm afraid that in reading the examples provided it is difficulties for me not simply to think that EVERY SINGLE ONE of them would be FAR easier to read if it were an `assert` instead. The API of the library is a bit noisy, but I think the obstacle it's more in the higher level design for me. Adding many layers of expensive runtime checks and many lines of code in order to assure simple predicates that a glance at the code or unit tests would do better seems wasteful. I just cannot imagine wanting to write or work on the kind of codebase that is down here. If some people or organizations want to come in this manner, sure a library is great. But I definitely don't want it in the syntax, nor even in the standard library. On Sat, Sep 15, 2018, 2:53 AM Marko Ristin-Kaufmann <marko.ristin@gmail.com> wrote:
On Sep 15, 2018 2:53 AM, "Marko Ristin-Kaufmann" <marko.ristin@gmail.com> wrote: Hi, Let me make a couple of practical examples from the work-in-progress ( https://github.com/Parquery/pypackagery, branch mristin/initial-version) to illustrate again the usefulness of the contracts and why they are, in my opinion, superior to assertions and unit tests. What follows is a list of function signatures decorated with contracts from pypackagery library preceded by a human-readable description of the contracts. The invariants tell us what format to expect from the related string properties. @icontract.inv(lambda self: self.name.strip() == self.name) @icontract.inv(lambda self: self.line.endswith("\n")) class Requirement: """Represent a requirement in requirements.txt.""" def __init__(self, name: str, line: str) -> None: """ Initialize. :param name: package name :param line: line in the requirements.txt file """ ... The postcondition tells us that the resulting map keys the values on their name property. @icontract.post(lambda result: all(val.name == key for key, val in result.items())) def parse_requirements(text: str, filename: str = '<unknown>') -> Mapping[str, Requirement]: """ Parse requirements file and return package name -> package requirement as in requirements.txt :param text: content of the ``requirements.txt`` :param filename: where we got the ``requirements.txt`` from (URL or path) :return: name of the requirement (*i.e.* pip package) -> parsed requirement """ ... The postcondition ensures that the resulting list contains only unique elements. Mind that if you returned a set, the order would have been lost. @icontract.post(lambda result: len(result) == len(set(result)), enabled=icontract.SLOW) def missing_requirements(module_to_requirement: Mapping[str, str], requirements: Mapping[str, Requirement]) -> List[str]: """ List requirements from module_to_requirement missing in the ``requirements``. :param module_to_requirement: parsed ``module_to_requiremnt.tsv`` :param requirements: parsed ``requirements.txt`` :return: list of requirement names """ ... Here is a bit more complex example. - The precondition A requires that all the supplied relative paths (rel_paths) are indeed relative (as opposed to absolute). - The postcondition B ensures that the initial set of paths (given in rel_paths) is included in the results. - The postcondition C ensures that the requirements in the results are the subset of the given requirements. - The precondition D requires that there are no missing requirements (*i.e. *that each requirement in the given module_to_requirement is also defined in the given requirements). @icontract.pre(lambda rel_paths: all(rel_pth.root == "" for rel_pth in rel_paths)) # A @icontract.post( lambda rel_paths, result: all(pth in result.rel_paths for pth in rel_paths), enabled=icontract.SLOW, description="Initial relative paths included") # B @icontract.post( lambda requirements, result: all(req.name in requirements for req in result.requirements), enabled=icontract.SLOW) # C @icontract.pre( lambda requirements, module_to_requirement: missing_requirements(module_to_requirement, requirements) == [], enabled=icontract.SLOW) # D def collect_dependency_graph(root_dir: pathlib.Path, rel_paths: List[pathlib.Path], requirements: Mapping[str, Requirement], module_to_requirement: Mapping[str, str]) -> Package: """ Collect the dependency graph of the initial set of python files from the code base. :param root_dir: root directory of the codebase such as "/home/marko/workspace/pqry/production/src/py" :param rel_paths: initial set of python files that we want to package. These paths are relative to root_dir. :param requirements: requirements of the whole code base, mapped by package name :param module_to_requirement: module to requirement correspondence of the whole code base :return: resolved depedendency graph including the given initial relative paths, """ I hope these examples convince you (at least a little bit :-)) that contracts are easier and clearer to write than asserts. As noted before in this thread, you can have the same *behavior* with asserts as long as you don't need to inherit the contracts. But the contract decorators make it very explicit what conditions should hold *without* having to look into the implementation. Moreover, it is very hard to ensure the postconditions with asserts as soon as you have a complex control flow since you would need to duplicate the assert at every return statement. (You could implement a context manager that ensures the postconditions, but a context manager is not more readable than decorators and you have to duplicate them as documentation in the docstring). In my view, contracts are also superior to many kinds of tests. As the contracts are *always* enforced, they also enforce the correctness throughout the program execution whereas the unit tests and doctests only cover a list of selected cases. Furthermore, writing the contracts in these examples as doctests or unit tests would escape the attention of most less experienced programmers which are not used to read unit tests as documentation. Finally, these unit tests would be much harder to read than the decorators (*e.g.*, the unit test would supply invalid arguments and then check for ValueError which is already a much more convoluted piece of code than the preconditions and postconditions as decorators. Such testing code also lives in a file separate from the original implementation making it much harder to locate and maintain). Mind that the contracts *do not* *replace* the unit tests or the doctests. The contracts make merely tests obsolete that test that the function or class actually observes the contracts. Design-by-contract helps you skip those tests and focus on the more complex ones that test the behavior. Another positive effect of the contracts is that they make your tests deeper: if you specified the contracts throughout the code base, a test of a function that calls other functions in its implementation will also make sure that all the contracts of that other functions hold. This can be difficult to implement with standard unit test frameworks. Another aspect of the design-by-contract, which is IMO ignored quite often, is the educational one. Contracts force the programmer to actually sit down and think *formally* about the inputs and the outputs (hopefully?) *before* she starts to implement a function. Since many schools use Python to teach programming (especially at high school level), I imagine writing contracts of a function to be a very good exercise in formal thinking for the students. Please let me know what points *do not *convince you that Python needs contracts (in whatever form -- be it as a standard library, be it as a language construct, be it as a widely adopted and collectively maintained third-party library). I would be very glad to address these points in my next message(s). Cheers, Marko _______________________________________________ Python-ideas mailing list Python-ideas@python.org https://mail.python.org/mailman/listinfo/python-ideas Code of Conduct: http://python.org/psf/codeofconduct/
I just want to point out that you don't need permission from anybody to start a library. I think developing and popularizing a contracts library is a reasonable goal -- but that's something you can start doing at any time without waiting for consensus. And if it gets popular enough, maybe it'll be added to the standard library in some form. That's what happened with attrs, iirc -- it got fairly popular and demonstrated there was an unfilled niche, and so Python acquired dataclasses.. The contracts make merely tests obsolete that test that the function or
class actually observes the contracts.
Is this actually the case? Your contracts are only checked when the function is evaluated, so you'd still need to write that unit test that confirms the function actually observes the contract. I don't think you necessarily get to reduce the number of tests you'd need to write. Please let me know what points *do not *convince you that Python needs
contracts
While I agree that contracts are a useful tool, I don't think they're going to be necessarily useful for *all* Python programmers. For example, contracts aren't particularly useful if you're writing fairly straightforward code with relatively simple invariants. I'm also not convinced that libraries where contracts are checked specifically *at runtime* actually give you that much added power and impact. For example, you still need to write a decent number of unit tests to make sure your contracts are being upheld (unless you plan on checking this by just deploying your code and letting it run, which seems suboptimal). There's also no guarantee that your contracts will necessarily be *accurate*. It's entirely possible that your preconditions/postconditions might hold for every test case you can think of, but end up failing when running in production due to some edge case that you missed. (And if you decide to disable those pre/post conditions to avoid the efficiency hit, you're back to square zero.) Or I guess to put it another way -- it seems what all of these contract libraries are doing is basically adding syntax to try and make adding asserts in various places more ergonomic, and not much else. I agree those kinds of libraries can be useful, but I don't think they're necessarily useful enough to be part of the standard library or to be a technique Python programmers should automatically use by default. What might be interesting is somebody wrote a library that does something more then just adding asserts. For example, one idea might be to try hooking up a contracts library to hypothesis (or any other library that does quickcheck-style testing). That might be a good way of partially addressing the problems up above -- you write out your invariants, and a testing library extracts that information and uses it to automatically synthesize interesting test cases. (And of course, what would be very cool is if the contracts could be verified statically like you can do in languages like dafny -- that way, you genuinely would be able to avoid writing many kinds of tests and could have confidence your contracts are upheld. But I understanding implementing such verifiers are extremely challenging and would probably have too-steep of a learning curve to be usable by most people anyways.) -- Michael On Fri, Sep 14, 2018 at 11:51 PM, Marko Ristin-Kaufmann < marko.ristin@gmail.com> wrote:
Hi David Maertz and Michael Lee, Thank you for raising the points. Please let me respond to your comments in separation. Please let me know if I missed or misunderstood anything. *Assertions versus contracts.* David wrote:
I think there are two misunderstandings on the role of the contracts. First, they are part of the function signature, and not of the implementation. In contrast, the assertions are part of the implementation and are completely obscured in the signature. To see the contracts of a function or a class written as assertions, you need to visually inspect the implementation. The contracts are instead engraved in the signature and immediately visible. For example, you can test the distinction by pressing Ctrl + q in Pycharm. Second, assertions are only suitable for preconditions. Postconditions are practically unmaintainable as assertions as soon as you have multiple early returns in a function. The invariants implemented as assertions are always unmaintainable in practice (except for very, very small classes) -- you need to inspect each function of the class and all their return statements and manually add assertions for each invariant. Removing or changing invariants manually is totally impractical in my view. *Efficiency and Evidency. *David wrote:
I'm not very sure what you mean by expensive runtime checks -- every single contract can be disabled at any point. Once a contract is disabled, there is literally no runtime computational cost incurred. The complexity of a contract during testing is also exactly the same as if you wrote it in the unit test. There is a constant overhead due to the extra function call to check the condition, but there's no more time complexity to it. The overhead of an additional function call is negligible in most practical test cases. When you say "a glance at the code", this implies to me that you referring to your own code and not to legacy code. In my experience, even simple predicates are often not obvious to see in other people's code as one might think (*e.g. *I had to struggle with even most simple ones like whether the result ends in a newline or not -- often having to actually run the code to check experimentally what happens with different inputs). Postconditions prove very useful in such situations: they let us know that whenever a function returns, the result must satisfy its postconditions. They are formal and obvious to read in the function signature, and hence spare us the need to parse the function's implementation or run it. Contracts in the unit tests.
(emphasis mine) Defining contracts in a unit test is, as I already mentioned in my previous message, problematic due to two reasons. First, the contract resides in a place far away from the function definition which might make it hard to find and maintain. Second, defining the contract in the unit test makes it impossible to put the contract in the production or test it in a call from a different function. In contrast, introducing the contract as a decorator works perfectly fine in all the three above-mentioned cases (smoke unit test, production, deeper testing). *Library. *Michael wrote:
As a matter of fact, I already implemented the library which covers most of the design-by-contract including the inheritance of the contracts. (The only missing parts are retrieval of "old" values in postconditions and loop invariants.) It's published on pypi as "icontract" package (the website is https://github.com/Parquery/icontract/). I'd like to gauge the interest before I/we even try to make a proposal to make it into the standard library. The discussions in this thread are an immense help for me to crystallize the points that would need to be addressed explicitly in such a proposal. If the proposal never comes about, it would at least flow into the documentation of the library and help me identify and explain better the important points. *Observation of contracts. *Michael wrote:
Assuming that a contracts library is working correctly, there is no need to test whether a contract is observed or not -- you assume it is. The same applies to any testing library -- otherwise, you would have to test the tester, and so on *ad infinitum.* You still need to evaluate the function during testing, of course. But you don't need to document the contracts in your tests nor check that the postconditions are enforced -- you assume that they hold. For example, if you introduce a postcondition that the result of a function ends in a newline, there is no point of making a unit test, passing it some value and then checking that the result value ends in a newline in the test. Normally, it is sufficient to smoke-test the function. For example, you write a smoke unit test that gives a range of inputs to the function by using hypothesis library and let the postconditions be automatically checked. You can view each postcondition as an additional test case in this scenario -- but one that is also embedded in the function signature and also applicable in production. Not all tests can be written like this, of course. Dealing with a complex function involves writing testing logic which is too complex to fit in postconditions. Contracts are not a panacea, but they absolute us from implementing trivial testing logic while keeping the important bits of the documentation close to the function and allowing for deeper tests. *Accurate contracts. *Michael wrote:
Unfortunately, there is no practical exit from this dilemma -- and it applies all the same for the tests. Who guarantees that the testing logic of the unit tests are correct? Unless you can formally prove that the code does what it should, there is no way around it. Whether you write contracts in the tests or in the decorators, it makes no difference to accuracy. If you missed to test an edge case, well, you missed it :). The design-by-contract does not make the code bug-free, but makes the bugs *much less likely* and *easier *to detect *early*. In practice, if there is a complex contract, I encapsulate its complex parts in separate functions (often with their own contracts), test these functions in separation and then, once the tests pass and I'm confident about their correctness, put them into contracts. (And if you decide to disable those pre/post conditions to avoid the
efficiency hit, you're back to square zero.)
In practice, we at Parquery AG let the critical contracts to run in production to ensure that the program blows up before it exercises undefined behavior in a critical situation. The informative violation errors of the icontract library help us to trace the bugs more easily since the relevant values are part of the error log. However, if some of the contracts are too inefficient to check in production, alas you have to turn them off and they can't be checked since they are inefficient. This seems like a tautology to me -- could you please clarify a bit what you meant? If a check is critical and inefficient at the same time then your problem is unsolvable (or at least ill-defined); contracts as well as any other approach can not solve it. *Ergonimical assertions. *Michael wrote:
Please do not underestimate another aspect of the contracts, namely the value of contracts as verifiable documentation. Please note that the only alternative that I observe in practice without design-by-contract is to write contracts in docstrings in *natural language*. Most often, they are just assumed, so the next programmer burns her fingers expecting the contracts to hold when they actually differ from the class or function description, but nobody bothered to update the docstrings (which is a common pitfall in any code base over a longer period of time). *Automatic generation of tests.* Michael wrote:
This is the final goal and my main motivation to push for design-by-contract in Python :). There is a whole research community that tries to come up with automatic test generations, and contracts are of great utility there. Mind that generating the tests based on contracts is not trivial: hypothesis just picks elements for each input independently which is a much easier problem. However, preconditions can define how the arguments are *related*. Assume a function takes two numbers as arguments, x and y. If the precondition is y < x < (y + x) * 10, it is not trivial even for this simple example to come up with concrete samples of x and y unless you simply brute-force the problem by densely sampling all the numbers and checking the precondition. I see a chicken-and-egg problem here. If design-by-contract is not widely adopted, there will also be fewer or no libraries for automatic test generation. Honestly, I have absolutely no idea how you could approach automatic generation of test cases without contracts (in one form or the other). For example, how could you automatically mock a class without knowing its invariants? Since generating test cases for functions with non-trivial contracts is hard (and involves collaboration of many people), I don't expect anybody to start even thinking about it if the tool can only be applied to almost anywhere due to lack of contracts. Formal proofs and static analysis are even harder beasts to tame -- and I'd say the argument holds true for them even more. David and Michael, thank you again for your comments! I welcome very much your opinion and any follow-ups as well as from other participants on this mail list. Cheers, Marko On Sat, 15 Sep 2018 at 10:42, Michael Lee <michael.lee.0x2a@gmail.com> wrote:
Hi, Again a brief update. * icontract supports now static and class methods (thanks to my colleague Adam Radomski) which came very handy when defining a group of functions as an interface *via* an abstract (stateless) class. The implementors then need to all satisfy the contracts without needing to re-write them. You could implement the same behavior with *_impl or _* ("protected") methods where public methods would add the contracts as asserts, but we find the contracts-as-decorators more elegant (N functions instead of 2*N; see the snippet below). * We implemented a linter to statically check that the contract arguments are defined correctly. It is available as a separate Pypi package pyicontract-lint (https://github.com/Parquery/pyicontract-lint/). Next step will be to use asteroid to infer that the return type of the condition function is boolean. Does it make sense to include PEX in the release on github? * We plan to implement a sphinx plugin so that contracts can be readily visible in the documentation. Is there any guideline or standard/preferred approach how you would expect this plugin to be implemented? My colleagues and I don't have any experience with sphinx plugins, so any guidance is very welcome. class Component(abc.ABC, icontract.DBC): """Initialize a single component.""" @staticmethod @abc.abstractmethod def user() -> str: """ Get the user name. :return: user which executes this component. """ pass @staticmethod @abc.abstractmethod @icontract.post(lambda result: result in groups()) def primary_group() -> str: """ Get the primary group. :return: primary group of this component """ pass @staticmethod @abc.abstractmethod @icontract.post(lambda result: result.issubset(groups())) def secondary_groups() -> Set[str]: """ Get the secondary groups. :return: list of secondary groups """ pass @staticmethod @abc.abstractmethod @icontract.post(lambda result: all(not pth.is_absolute() for pth in result)) def bin_paths(config: mapried.config.Config) -> List[pathlib.Path]: """ Get list of binary paths used by this component. :param config: of the instance :return: list of paths to binaries used by this component """ pass @staticmethod @abc.abstractmethod @icontract.post(lambda result: all(not pth.is_absolute() for pth in result)) def py_paths(config: mapried.config.Config) -> List[pathlib.Path]: """ Get list of py paths used by this component. :param config: of the instance :return: list of paths to python executables used by this component """ pass @staticmethod @abc.abstractmethod @icontract.post(lambda result: all(not pth.is_absolute() for pth in result)) def dirs(config: mapried.config.Config) -> List[pathlib.Path]: """ Get directories used by this component. :param config: of the instance :return: list of paths to directories used by this component """ pass On Sat, 15 Sep 2018 at 22:14, Marko Ristin-Kaufmann <marko.ristin@gmail.com> wrote:
Hi, I implemented a sphinx extension to include contracts in the documentation: https://github.com/Parquery/sphinx-icontract The extension supports inheritance. It lists all the postconditions and invariants including the inherited one. The preconditions are grouped by classes with ":requires:" and ":requires else:". I was unable to get the syntax highlighting for in-line code to work -- does anybody know how to do that in Sphinx? The results can be seen, *e.g.* in this documentation: https://pypackagery.readthedocs.io/en/latest/packagery.html On a more general note: is there any blocker left why you would *not *use the contracts in your code? Anything I could improve or fix in icontract that would make it more convincing to use (apart from implementing static contract checking and automatic test generation :))? Cheers, Marko On Thu, 20 Sep 2018 at 22:52, Marko Ristin-Kaufmann <marko.ristin@gmail.com> wrote:
You might also be interested in pep-563 <https://www.python.org/dev/peps/pep-0563/>. although it is not intended for design by contract, it can help (syntactically). Elazar On Wed, Aug 15, 2018 at 11:07 PM Marko Ristin-Kaufmann < marko.ristin@gmail.com> wrote:
Hi Marko Thank you for introducing yourself, and clearly stating your question. That helps us all. You asked:
Could somebody update me on the state of the discussion on this matter?
I think bring the existing PEP up to date would be a good starting point. Its content hasn't been changed since 2003 (except for PEP-wide admin changes. (Recall that Python 3.0 was released in 2008.) https://www.python.org/dev/peps/pep-0316/ https://github.com/python/peps/commits/master/pep-0316.txt In fact, revising the PEP might be enough to answer your question. What do you think, Marko? Experts: is there a process for revising old PEPs, such as this one? Or at least a precedent we could follow (or adapt)? -- Jonathan
On Thu, 16 Aug 2018 at 10:41, Jonathan Fine <jfine2358@gmail.com> wrote:
I'm not aware of a formal process, but I'd have thought the following steps would be a reasonable approach: 1. Review the PEP, and research the discussions that happened at the time, particularly of interest is why the PEP was deferred. 2. Consider what (if anything) has changed since the original deferral (which could simply be "time has moved on, people's views may have changed" but ideally would include a bit more in the way of concrete motivation). 3. Contact the original PEP author and ask if he is interested in reopening the discussion, collaborating on a revision, or handing the PEP over. 4. Start up a discussion here, pointing out the original PEP and summarising the previous debate and why you want to restart the discussion. If you're hoping to change the details of the original PEP, summarise your changes and why you feel they are an improvement over the original. To answer the OP's question more directly:
Could somebody update me on the state of the discussion on this matter?
As far as I am aware, there has been no discussion on this subject since the PEP 316 discussions which ended up in its deferral. Elazar mentioned PEP 563, and there *may* have been mention of design by contract uses in the discussions on that PEP, but you'd have to search the mailing list archives to confirm that one way or another. Hence the suggestions that if you want to restart discussion, reviving PEP 316 is likely the best approach. Paul
Hi Jonathan and Paul, Thank you very much for your suggestions! I will try to contact the author of the PEP. Let me clarify a bit a potential misunderstanding. Please mind that contracts are not tied to individual variables, but to expressions. Think of it as defining a lambda which takes as input all the arguments of the function (and a result variable in case of post-conditions) which always needs to evaluate to True. Cheers, Marko Le jeu. 16 août 2018 à 12:24, Paul Moore <p.f.moore@gmail.com> a écrit :
I think that annotations were suggested because you could write an expression there without getting evaluated. I've thought about this problem many times in the past (as a Python dev with a long history working in Eiffel too).... For me one of the crucial issue that is hard to translate into the python model is that the assertions (say, a function precondition) are not conceptually part of the function itself, but the interface of the class. The "natural" python ways of attaching these assertions somehow to the function object do not work when you also use inheritance, because when you override a method the new function object clobbers the previous one. I've experimented at some point on how to put them in classes (and doing metaclass or __getattribute__ tricks) but nothing convinced me). In general, the way that python puts method call and inheritance semantic in a specific layout of runtime objects (which in general is really clever) seems to be a bit alien to the DbC idea where the asbtraction/interface of the class is conceptually separate and has independent information wrt to the runtime objects. On 16 August 2018 at 18:49, Marko Ristin-Kaufmann <marko.ristin@gmail.com> wrote:
-- <https://www.machinalis.com/> Daniel Moisset Technical Leader A: 1 Fore St, EC2Y 9DT London <https://goo.gl/maps/pH9BBLgE8dG2> P: +44 7398 827139 <+44+7398+827139> M: dmoisset@machinalis.com <dmoisset@machinalis.com> | S: dmoisset <http://www.linkedin.com/company/456525> <http://www.twitter.com/machinalis> <http://www.facebook.com/machinalis> <https://www.instagram.com/machinalis.life/>
pycontracts may be worth a look. https://andreacensi.github.io/contracts/ - @contract decorator, annotations, docstrings IDK if pycontracts supports runtime parameter validations that involve more than one parameter. Inheritance does appear to be supported, as are numpy array dimension constraints. I can't recall whether the pycontracts expression language precedes MyPy compile-time annotations; both with one syntax really would be great. On Monday, August 20, 2018, Daniel Moisset <dmoisset@machinalis.com> wrote:
Hi, I had a look at the messages related to the PEP in question (PEP 316) in the archive. As far as I can tell, the main objection is that you can achieve contracts by implementing it with decorators. I think that these objections miss what actually Daniel Moisset wrote in his message: contracts are more than pre- and post-condition checks on a function. The inheritance of decorators does not imply just inheriting the pre- and post-conditions, but also relaxing and tightening them (represented by "require else" and "ensure then" in Eiffel). If this is to be used effectively in practice with little overhead then we would either need to introduce new syntax to the language or make the compiler improve the byte code on the fly. Is there any chance to introduce these constructs in the language or is it too small a feature for such a big change? Since we wanted to have contracts in Golang, we implemented a tool that synchronizes the documentation of a function with the function code ( https://github.com/Parquery/gocontracts). Maybe this is the easier path to follow in Python as well? @Wes Turner: thanks for pointing to pycontracts. I'm aware of the library. It implements only contracts based on a single property. We found that limiting and rolled out our own solution that suited us much better: https://github.com/Parquery/icontract/ I also found informative messages on contract breach to be particularly important for fast development and error inspection in the production. Cheers, Marko On 21 August 2018 at 04:44, Wes Turner <wes.turner@gmail.com> wrote:
On Tue, Aug 21, 2018 at 09:06:54AM +0200, Marko Ristin-Kaufmann wrote:
Is there any chance to introduce these constructs in the language or is it too small a feature for such a big change?
I don't think contracts is a small feature. I think it is a HUGE feature, but under-appreciated by most developers. (Probably due to unfamiliarity, and the difficulty in using it in a language with no syntactic support.) Whether it is practical to add it to Python, I don't know. I suspect that we would have to develop some sort of third-party solution first, even if it did not do everything contracts ought to do (or do them less efficiently) as a proof of concept. PyContracts is probably a good place to start. For those who missed Wes' email: https://andreacensi.github.io/contracts/ Cobra is another good place to look, as it demonstrates a nice syntax that reads like a cross between Python and Eiffel: http://cobra-language.com/trac/cobra/wiki/Contracts -- Steve
On Tue, 21 Aug 2018 at 11:27, Steven D'Aprano <steve@pearwood.info> wrote:
Agreed. And it sounds like there are a lot of subtleties to contracts that I certainly hadn't appreciated (I don't know if the same is true of others). For example, On Tue, 21 Aug 2018 at 08:08, Marko Ristin-Kaufmann <marko.ristin@gmail.com> wrote:
I think that these objections miss what actually Daniel Moisset wrote in his message: contracts are more than pre- and post-condition checks on a function. The inheritance of decorators does not imply just inheriting the pre- and post-conditions, but also relaxing and tightening them (represented by "require else" and "ensure then" in Eiffel). If this is to be used effectively in practice with little overhead then we would either need to introduce new syntax to the language or make the compiler improve the byte code on the fly.
I've no idea what "relaxing" and "tightening" of contracts involves, or how it would translate into Python. So I'd imagine that in order to put together a proposal for adding contracts to Python, you'd need to explain what contracts are, and how they work, to get past people's preconceptions that "they are just assertions". Otherwise, it's likely to be hard to persuade people of the benefits. Personally, I don't really think I can comment much further, precisely because it looks like I don't know enough about what contracts are and how they'd be used to contribute :-) Paul
Hi, @Paul Moore: thanks for pointing out that many people are not familiar with design-by-contract. I was not aware of that. Let me give you a very short introduction into contracts and what they are good for. I'll review some existing libraries and highlight what features we missed (and why we developed our own library). I will finally conclude with why all these solutions (including our own one) are not a replacement for a proper support of design-by-contract in the language. Design-by-Contract Design-by-contract was gradually introduced as a concept by Bertrand Meyer in the 1980ies to provide a formal and verifiable interface specification between the software components. Up to then the interfaces were usually defined formally in terms of abstract data types (think of records / structs and classes). He extended abstract data types by adding "contract" conditions that should hold at different points during the execution of a program. The contracts allow you to formally write down your expectations about the program (as opposed to writing it informally in documentation of a class or a function). Hence we can automatically test that they hold either statically or during the execution of the program. This gives us many-fold benefits: - contracts prevent code rot (since they can be checked by a compiler or a static analysis tool such as mypy), - allow us to have much more sophisticated automatic generation of unit tests and automatic formal proofs of a program, - make the documentation explicit, accurate and verifiable and - accelerate the development since errors are caught early. The contracts are categorized as follows depending on at which point in the program they are verified: - Preconditions are contracts that should hold before the execution of a function. The *caller* is responsible to fulfill the preconditions. - Postconditions are contracts that should hold after the execution of a function. The *callee* is responsible to fulfill the postconditions. - Invariants should hold throughout the execution of the program. There are two types of invariants: loop invariants and class invariants. - Loop invariants should hold before and after each iteration step of a loop. - Class invariants should hold throughout the life time of a class ( *i.e.* between the calls to the public methods). The class invariants are suspended during the construction of an instance and in private methods to avoid cycles. You can think of the constructor method and public methods being responsible to fulfill the class invariants. The concept of design-by-contract is not limited only to concrete classes, but can be also applied to class hierarchies. Preconditions, postconditions and invariants are inherited. They can be also modified in the following ways: - The child class needs to fulfill all the invariants of its antecedent classes and its own ones. - The preconditions of a function of a child class can "weaken" or "relax" the preconditions of the parent class. In other words, it needs to fulfill *either* the preconditions of the parent class *or *its own set of preconditions. This is reflected in Eiffel by using the keyword *require else.* - The postconditions of a a child class can "strengthen" or "tighten" the postconditions of the parent class. The function needs to fulfill all the postconditions of the parent class' function *and *its own set of postconditions. In Eiffel, this is designated with *ensure then* keyword. Invariants operate only on the values of instance properties. Preconditions operate on both function arguments and instance properties. Postconditions need yet one more instrument: they operate on function arguments, instance properties and the result of a function, but can access all these values both at their *old* state, before the function call, and their *new *state, after the function call. In Eiffel, you use the keyword *old* to indicate the value before the function call. Let me illustrate the concepts by adapting the examples from https://www.eiffel.org/doc/eiffel/ET-_Design_by_Contract_%28tm%29%2C_Asserti... and https://www.eiffel.org/doc/eiffel/ET-_Inheritance. I will paraphrase the example code in Python assuming that invariant, require, ensure and old were introduced as keywords. *class *Account: *def *__init__(self, balance: int) -> *None*: self.balance = balance self.deposits = [] *# type: List[int] **def *update(self, sum: int)->*None*: require: sum >= 0 self.balance += sum self.deposits.append(sum) ensure: len(self.deposits) = len(old self.deposits) + 1, *"one more deposit" *ensure: self.balance = old self.balance + sum, *"updated" *invariant: *not *self.deposits *or *self.balance == sum(self.deposits), *"consistent balance" *invariant: self.deposits *or *self.balance == 0, *"zero if no deposits" class *CheckingAccount(Account): *def *update(self, sum: int) -> *None*: require *else*: self.balance >= sum self.balance += sum self.deposits.append(sum) invariant: self.balance >= 0 For more examples and details, please see the before-mentioned two web pages on Eiffel. Current Implementations A series of open-sourced libraries have been developed to bring design-by-contract to Python. *PyContracts* (https://pypi.org/project/PyContracts/) introduced the preconditions and postconditions as special annotations: @contract def my_function(a : 'int,>0', b : 'list[N],N>0') -> 'list[N]': # Requires b to be a nonempty list, and the return # value to have the same length. ... new_contract('valid_name', lambda s: isinstance(s, str) and len(s)>0) @contract(names='dict(int: (valid_name, int))') def process_accounting(records): ... I personally find these annotations hard to read. The contracts seem mostly focused on type checks and on single arguments. Moreover, without support in IDEs and static analysis tools, annotations are susceptible to code rot if not verified by a linter (as far as I could find out, there are not linters that check pycontracts -- please correct me if I'm wrong). I also don't see how you could elegantly implement a check based on multiple arguments (*e.g., *sum >= balance in the CheckingAccount example). *Dpcontracts* (https://pypi.org/project/dpcontracts/) is a library based on decorators. It encapsulates the arguments of a function and its result as the arguments of the condition function:
It also supports invariants:
Contracts (https://pypi.org/project/contracts/), pyadbc ( https://pypi.org/project/pyadbc/) and pcd (https://pypi.org/project/pcd/) are similar in terms of features to dpcontracts, but seem not to be maintained any more. We found that all the presented methods were a bit impractical to use in everyday programming. Pycontracts forces the programmer to learn a new syntax which is not statically checked. While we liked the features provided by dpcontracts, we found that exception messages thrown at contract breach were uninformative. The programmer is forced to repeat every condition in text if s/he is to make any use of the error once it happens. This can lead to mismatch between the messages and code (similar how the comments tend to rot) and can make debugging and tracing bugs very hard. Moreover, it is tedious and repetitive to document the conditions twice which makes programmers sometimes reluctant to adopt it and apply it widely. We therefore developed *icontract* (https://pypi.org/project/icontract/). We decided to leave out class invariants for practical reasons (we rarely use classes in our applications and we needed conditions in production for which the invariants are often impractical), but invariants can be easily added if there is a demand. icontract is based on decorators and uses lambda functions for the conditions that match the argument names of the function. The argument *result* is reserved for the result of the function in postconditions. Here is the example usage:
Mind that there is no mandatory description in the contract yet the message is informative. We achieve that by re-executing the condition function and tracing the values by examining its abstract syntax tree. The re-computation can also deal with more complex expressions and outer scope:
We found informative messages to be a tremendous booster when debugging since you often immediately see what caused the contract breach and not only that it was broken. This often points you directly to the cause of the bug. Moreover, conditions are much more readable when not cluttered by redundant descriptions. This is important when you use them as part of the documentation that you inspect with an IDE (*e.g., *in PyCharm). By our subjective impression, other solutions resulted in hard-to-read documentation in PyCharm. While descriptions can also be added to the icontract conditions, we rarely find that necessary since most conditions are self-explanatory. Insufficiencies of the Current Libraries All the libraries described here were not trivial to implement and come with a substantial computational overhead. In one of our benchmarks, we found that having a precondition made a function run at least 6x slower (we traced the slow-down to an additional function invocation which is costly in Python). I don't think that it is possible to implement *old *keyword for most practical applications since the execution would be even slower. I found no library so far that supports inheritance, strengthening (ensure then) and weakening (*require else)* of the contracts out-of-the-box. My intuition tells me that it is possible to implement such a feature in a library, but the implementation will definitely be very complex. Apart from computational efficiency and complexity of implementation, I also see the variety of libraries as a problem for the adoption of design-by-contract. With multiple solutions and each team having their own preferences there is a dilemma which solution to choose. Without a wide adoption of a single solution we can not expect an emergence of tools such as automatic test generators built on top of the contracts which is where the actual tremendous benefits really await. A standard solution would allow us to have uniform, widely-adopted and efficient design-by-contracts in Python which no library by itself can achieve. Sketch of a Solution Instead of introducing new keywords, Python could introduce a built-in module based on decorators. The interpreters could be extended such as that they in-line the code directly into the function whenever the decorators of this built-in module is encountered. This would substantially reduce the computational overhead while it would allow us to avoid changes to language syntax. However, before we even start looking at a solution, I see it necessary that we first discuss more to which degree contracts should be introduced to Python and what the use cases would look like. Cheers, Marko
Total noob speaking here, but.... Those contracts are mostly important during development right ? Slowdown isn't that much of an issue during development. So you could make a debug mode that enforces the contracts, and a production mode that code users can use during production to stop the slowdown - in this case, your decorators can return their functions/classes unaltered. If they do end up with problems, you can always ask them to run the same inputs with debug mode on to see where it goes wrong. For the rest, i'm totally new to design by contract. I do get the draw of it, but im not sure if I'd ever really use it. I tend to solve my problems with a royal sprinkling of logger.debug(f"{val_i_need_to_check}").
On Mon, Aug 27, 2018 at 09:24:20AM +0100, Ivan Levkivskyi wrote:
TBH, I think one of the main points of design by contract is that contracts are verified statically.
No, that's not correct. Contracts may be verified statically if the compiler is able to do so, but they are considered runtime checks. Static checks are an optimization. For example, the Eiffel docs describe one possible contract as "the graph contains no cycle" and can contain function calls. https://www.eiffel.org/doc/eiffel/ET-_Design_by_Contract_%28tm%29%2C_Asserti... -- Steven
On Mon, 27 Aug 2018 at 11:39, Steven D'Aprano <steve@pearwood.info> wrote:
Considered by whom? By people who prefer `assert isinstance(x, int)` over `x: int`? :-) Contract in 99% of cases is just another word for type (maybe a very complex type like `DAG[T] <: Graph[T]`). Everything else, like `x >= y` is better expressed as an explicit assert with an assert message. But again this is rather IMO, than any kind of definition. There is only one use case I see now where a dedicated syntax would give a large readability gain: something like `self.x >= self.y`. But on the other hand I think such situations are too rare to justify any _new_ syntax. -- Ivan
Ivan and Steve wrote
TBH, I think one of the main points of design by contract is that contracts are verified statically.
No, that's not correct.
https://www.eiffel.org/doc/eiffel/ET-_Design_by_Contract_%28tm%29%2C_Asserti...
The page from Steve supplied (URL above) states
Based on the Eiffel docs, I find Ivan's opinion reasonable. He said it was "one of the main points". The goal is to detect errors immediately. Run-time assertion monitoring and static typing are two means towards that end. Our shared problem and goal is to have similar immediate detection of errors in Python (when the development process requires that degree of rigour). -- Jonathan
I didn't want to embarass Ivan any further by seemingly picking on his opinion about contracts being always statically checked, but when I asked off-list I got told to reword and post it here. So here it is. Sorry Ivan if this makes you feel I'm picking on you, that isn't my intention. On Mon, Aug 27, 2018 at 12:25:44PM +0100, Jonathan Fine wrote: [..]
Ivan said that static checking was a main point. Those Eiffel docs which you (Jonathon) quoted approvingly describe them as "run-time assertions". You describe them as "run-time assertions". I described them as "run-time assertions". So I'm having difficulty in understand what part of Ivan's opinion that they are compile-time static checks is "reasonable". If there's something I'm missing about Ivan's comment that you can see, I'd like to be enlightened. I don't see the relevance of the "two means towards that end" -- we have many means towards detecting bugs as early as possible: - correctness proofs - test-driven development - type checking - design by contract (and possibly more). If it was just a throw-away comment and I'm reading more into it than you intended, that's okay too, but I'd like to understand what you meant.
Well, yes, but what's that got to do with the question of whether contracts are checked statically or at runtime? -- Steve
This is about a difference of opinion regarding design by contract and static checking, that Steve D'Aprano has re-raised. Steve wrote that Ivan Levkivskyi's opinion was that:
contracts [are] always statically checked
This is what Ivan wrote:
TBH, I think one of the main points of design by contract is that contracts are verified statically.
There's no 'always' or 'all' here. I read it to mean 'sometimes' or 'some'. And also, that static verification is a good thing. My message of support for Ivan quoted the Eiffel docs.
https://www.eiffel.org/doc/eiffel/ET-_Design_by_Contract_%28tm%29%2C_Asserti...
I then wrote:
Based on the Eiffel docs, I find Ivan's opinion reasonable.
Steve wrote:
I'm sorry, but I'm just not seeing that. Either in what I quoted, or elsewhere in the page. True, my quote is from a section titled "Run-time assertion monitoring", but the context to me makes it clear that in Eiffel static typing IS NOT regarded as a run-time assertion. By the way, Steve wrote
when I asked off-list I got told to reword and post it here.
I don't think that's quite right. What I said off-list to Steve was
If you post [...] to python-ideas, I'd be happy to respond there.
It was my intention that it was up to Steve, whether or not to re-raise the issue. And now I'm doing my part of the bargain, by responding happily. I'm now happy to let this particular topic rest. -- Jonathan
On Wed, Aug 29, 2018 at 02:26:54PM +0100, Jonathan Fine wrote:
Fair enough, I should not have added "always" in my description. But you probably shouldn't have skipped the part I wrote earlier: "Contracts may be verified statically if the compiler is able to do so, but they are considered runtime checks. Static checks are an optimization." In context, you quoted me disagreeing with the "static" claim, but trimmed out my qualification that contracts may sometimes be statically verified when possible. [...]
Of course it isn't. By definition, static typing is done *statically*, at compile-time, not run-time. That has not been questioned and nobody has asserted that static typing is done at run-time. We're discussing whether *contracts* are checked at run-time. Because contracts are (in general) run-time assertions, they are a good fit for Python's execution model -- or at least they would be if we can find a good way to apply contracts to methods and classes. If they were static, they would be a bad fit and would probably need to be handled by a separate static checker, like MyPy does for static type checks. -- Steve
Jonathan Fine wrote:
I think you're misinterpreting the Eiffel docs here. It's saying that contracts *together* with static typing help to catch a lot of errors early in the development process. It's not saying that contracts are verified statically, or that all the errors thus caught are caught at compile time.
the context to me makes it clear that in Eiffel static typing IS NOT regarded as a run-time assertion.
That's true, but static typing and contracts are *different things* in Eiffel. Static types are checked at compile time, contracts are checked at run time. -- Greg
Hi, I think we got entangled in a discussion about whether design-by-contract is useful or not. IMO, the personal experience ("I never used/needed this feature") is quite an inappropriate rule whether something needs to be introduced into the language or not. There seems to be evidence that design-by-contract is useful. Let me cite Bertrand Meyer from his article "Why not program right?" that I already mentioned before:
6. Bertrand Meyer, Object-Oriented Software Construction, 2nd edition,
Is there any logical or empirical objection that the design-by-contract is not useful and hence does not merit to be introduced into the core language? There is little point in writing a PEP and fleshing out the details if the community will reject it on grounds that design-by-contract by itself is meaningless. So I'd suggest we clarify this first before we move on. @David Mertz:
Could you please elaborate a bit? I don't see how the annotations would make the contracts invoked on inheritance. Consider this simple case: class A: @icontract.pre(lambda x: x > 0) def some_method(self, x: int)->None: pass class B(A): # Precondition should be inherited here. def some_method(self, x: int) -> None: pass You would still need to somehow decorate manually the overridden methods even though you would not specify any new contracts, right? Is there a mechanism in Python that I am not aware of that would allow us to accomplish that? On Wed, 29 Aug 2018 at 22:41, Greg Ewing <greg.ewing@canterbury.ac.nz> wrote:
On Wed, Aug 29, 2018 at 3:07 PM Marko Ristin-Kaufmann < marko.ristin@gmail.com> wrote:
A metaclass does this pretty easily. I have a thing I wrote years ago called MIS (Multi-Inheritance Safety) that is used to ensure you don't do anything stupid in our physics-modeling database. The database is a collection of ~200 Python classes/mixins all deriving madly from each other to get various behaviors (Has mass? You'll need this. Has moments? You'll need this. Has physical extent, i.e., can be seen? You'll need these...). Anyhow, the basemost class has the metaclass, which traverses all the methods in the subclasses and makes sure you don't have methods with the same name in two distinct superclass trees and such things. It also forces you to be explicit when you overload a method from a base class (among other things, but this one is easy to illustrate). class MIS(type): def __init__(cls, name, bases, namespace): mro = cls.mro()[1:-1] # All the stuff between new class and 'object' for method_name, method in namespace.items(): if isinstance(method, executable_types): if not getattr(method, '_its_ok', False): # Make sure it's not in any baser class. # Could easily check for decorated pre/post properties and copy them... def override(func): # Mark the function as a valid override... func._its_ok = True return func class Base(metaclass=MIS): def something(self): pass class Derived(Base): @override # Signal that this is ok, otherwise we get an error. def something(self): ...
The technique Eric suggests is probably better than what I had in mind. But I was thinking you could have an "inherit" decorator for methods (or for a class as a whole). It's easy enough for a decorator to attach a `.__contracts__` attribute to either the class or the individual methods. Then the decorator(s) in the child can simply look through the `.__mro__` to find any such parent contracts. E.g.: class B(A): @inherit_invariants def some_method(self, x: int) -> None: pass @precondition(lambda x: x=42, inherit_parent=True) def other_method(self, x: int) -> float: return 42/5 I'm not writing a library to do this, so you can tweak the API to be different than my example. But this is already well possible. On the broader idea: Is there any logical or empirical objection that the design-by-contract is not useful and hence does not merit to be introduced into the core language? There is little point in writing a PEP and fleshing out the details if the community will reject it on grounds that design-by-contract by itself is meaningless. Lots of people have explained this relative to lots of ideas, probably mostly better than I will. Adding a new feature, even if it is *technically* backwards compatible, has HUGE costs. All the documentation—the books, articles, blog posts, videos, webinars, tutorials, etc.—about Python has to be updated. We get a divide between "code that will work in Python 3.9" versus what will run in 3.7. The cognitive burden of learning Python is increased for everyone in the world (millions of people) because even if they do not use a feature they will encounter code that does. There is another section of code in the implementation(s) of Python that can potentially have bugs and needs to be maintained by someone. Obviously, design-by-contract is not *meaningless*! It's a specific feature that is relatively well defined as a concept (and precisely defined in regard to Eiffel specifically; but we might not implement those *exact* semantics). It's also a feature that no languages in particularly widespread use have decided to implement at the language level. I've chatted with Meyer; he's definitely very smart and definitely strongly opinionated, but I also think he's wrong about the overall importance of this feature versus lots of others. In my mind, this feature doesn't come close to meeting the burden of those high costs listed above (and others I did not mention). But I don't have any say in what the core developers will do, beyond in that they might be influenced by my opinion here. Yours, David... On Wed, Aug 29, 2018 at 6:41 PM Eric Fahlgren <ericfahlgren@gmail.com> wrote:
-- Keeping medicines from the bloodstreams of the sick; food from the bellies of the hungry; books from the hands of the uneducated; technology from the underdeveloped; and putting advocates of freedom in prisons. Intellectual property is to the 21st century what the slave trade was to the 16th.
On 8/29/2018 6:39 PM, Eric Fahlgren wrote:
I think a metaclass is a non-starter. If one were used, it would preclude using contracts in any case where a metaclass were already used, or where one was needed in the future. I'm sure people will disagree with me on this. But, I think a more productive line of thinking is: what could be added to the language that would let contracts be implementable, and could also be used for other things, too? Sort of like how PEP 487 adds customizability that has wide applicability. Eric
On Wed, 29 Aug 2018 at 23:08, Marko Ristin-Kaufmann <marko.ristin@gmail.com> wrote:
Hi, I think we got entangled in a discussion about whether design-by-contract is useful or not. IMO, the personal experience ("I never used/needed this feature") is quite an inappropriate rule whether something needs to be introduced into the language or not.
It's not a key factor, but it is indicative, in the sense that if no-one has ever needed the feature *in Python*, then it's possibly not a good fit for how the language is used in real life. That doesn't exclude the possibility of a new feature offering a new and not previously considered technique, but if that's the assertion, then it's up to the individual proposing the new feature to persuade the community that there's a real and significant benefit.
Is there any logical or empirical objection that the design-by-contract is not useful and hence does not merit to be introduced into the core language? There is little point in writing a PEP and fleshing out the details if the community will reject it on grounds that design-by-contract by itself is meaningless. So I'd suggest we clarify this first before we move on.
"It could be useful" is nowhere near a strong enough reason for changing the language. Even "it is demonstrably useful in other languages" would have a hard time. What you need to demonstrate is that it would be useful *in Python code*. A good way of doing that is by reviewing a significant body of real-life Python code (the standard library is a common choice, but a big project like Django or SQLAlchemy would also be reasonable) and demonstrating how the code could be *improved* by using the proposed new feature. Here, the key is that the change has to be an improvement - readability is one (somewhat difficult to assess objectively) criterion, as is ease of identifying bugs, or efficiency (less need to repeat complex expressions, for example). Conversely, any proposed new feature needs to address how it impacts people who *don't* want to use it. That can be as simple as backward compatibility issues such as adding new keywords (people using that keyword as a variable name will have to change their code) but can also address any general runtime impact in the interpreter (bookkeeping and management of new data structures for example, or changes to class implementation details). and how people who don't use the new feature will be impacted if they encounter the feature in code written by others, or in code reviews. Also, there's the question of teachability. A proposed new feature must address how it will be explained to new and existing Python users. And given the confusion we're seeing in this thread ("aren't these just runtime assertions?") that's something that contracts will definitely have to address. I'm not saying that the proposal needs to offer a full tutorial on design by contract, but at a minimum, it'll have to cover why contracts aren't just runtime assertions, and how the differences can be made clear in the documentation and by trainers. I hope that helps explain what you'll need to do if you want to take this proposal forward, and why you're getting pushback in areas that maybe seem incidental to you. Personally, I'm interested in the feature, but I'm not sure I'm interested enough to want it in Python. My main questions are 1. How exactly do these differ from simple assertions? I don't understand the hints about "relaxing" and "strengthening" contracts in subclasses, in particular I've no idea how I'd express that in actual syntax. 2. Are we talking here about adding checks that would run in production code? Wouldn't that slow code down? How do I deal with the conflict between wanting tighter checks but not wanting to pay the cost at runtime of checking things that should never go wrong (contracts, as I understand it, are for protecting against code bugs, that's why there's no facility for trapping them and producing "user friendly" error messages)? I use assertions very sparingly in production code for precisely that reason - why would I be more willing to use contracts? Paul
On Mon, Aug 27, 2018 at 12:12:22PM +0100, Ivan Levkivskyi wrote:
No, considered by Bertrand Meyer, the inventor of Design By Contract and the programming language Eiffel. Contracts are not limited to the things which static type-checkers are capable of testing, but can and do involve checks which are impossible or impractical to test at compile-time, like "today is Tuesday" or "the account balance is greater than the amount you are trying to withdraw".
Contracts ARE assertions. They are assertions about the input a method expects, not merely the type but the value, what result it intends to return, and the invariants of the class after the method is called. Like assertions, they are called at runtime, and can be disabled. Using contracts to enforce type-correctness wouild be silly in Eiffel, because Eiffel already has a powerful static type system. Sybtax-wise, if you're interested in what is possible in a Python-like language, you could do worse than check out Cobra: http://cobra-language.com/trac/cobra/wiki/Contracts -- Steve
On Mon, Aug 27, 2018 at 10:50 PM, Steven D'Aprano <steve@pearwood.info> wrote:
Sometimes "type" doesn't mean the same thing to the language and to the human. Suppose you're trying to create a Python script that replicates a C program; you might want to declare that a variable is not of type "integer" but type "32-bit unsigned integer", with wrap-around. Or, wording it another way: "integer modulo 2**32". Is that an assertion of type, or of type and value? As a precondition to a function, requiring that a parameter be an integer no less than zero and no greater than 4294967295 is, in a sense, checking its type and its value; but it's kinda just asserting its type. AIUI, that's what Ivan meant by "complex type" - something that determines the domain of valid values as well as the concrete type. ChrisA
On Mon, Aug 27, 2018 at 11:00:22PM +1000, Chris Angelico wrote:
It is making an assertion about the value of an instance of type "int". Its not a separate type requiring an explicit coercion or cast. Its just a non-negative int less than 2**32. If the compiler supports the concept of a 32-bit unsigned integer type, then of course we can change our implementation to use that type instead of our regular ints. But we can't expect to pass such a 32-bit unsigned integer to a function which expects a regular int unless they are duck-type compatible, or the compiler performs automatic coercions. (So-called "weak typing"). A better example is the one I gave earlier, of a graph with no cycles. There is a deep fundamental difference between a *statically checked* DAG with no cycles (a graph which can never contain a cycle because the compiler won't let you create one) and a *dynamically checked* DAG that merely has no cycles *now* (it may have had cycles earlier, and it might have cycles later, but right now it has none). These are very different semantics, and Eiffel's contracts support the second kind: runtime value checks. -- Steve
Runtime checks: data validation & code validation Compile-time checks: code validation What sort of data validation is appropriate for assert statements or contacts that may be skipped due to trading performance for more risk ('optimized out')? Checking the value of a Number? Checking that a large graph has no cycles? Checking that a database table exists and has the appropriate relations and constraints? assert statements are skipped at runtime with -O and -OO whether or not they're in [reorderable] aspects applied with decorators, at the beginning or end of a function, or in methods named something like setUp and tearDown. https://docs.python.org/3/using/cmdline.html#envvar-PYTHONOPTIMIZE https://docs.python.org/3/using/cmdline.html#cmdoption-o
On Monday, August 27, 2018, Steven D'Aprano <steve@pearwood.info> wrote:
On Mon, Aug 27, 2018 at 11:01:21AM -0400, Wes Turner wrote:
That depends on what you mean by "data validation". Testing for bad input, or testing for predictable error states such as I/O errors, missing files, permission errors, server down etc is not appropriate for assertions (which includes contracts). The rule I use is that assertions are for: (1) testing your program state, which is under your control; and (2) communicating the intention of your program as executable code rather than comments. The ultimate aim of contracts and assertions is to eventually disable them when the program is bug-free. The Eiffel docs say: It should be clear from the preceding discussion that contracts are not a mechanism to test for special conditions, for example erroneous user input. For that purpose, the usual control structures [...] are available [...] An assertion is instead a correctness condition governing the relationship between two software modules (not a software module and a human, or a software module and an external device). ... Bluntly: Rule -- Assertion Violation: A run-time assertion violation is the manifestation of a bug. https://www.eiffel.org/doc/eiffel/ET-_Design_by_Contract_%28tm%29%2C_Asserti... For detecting *external* error states (anything to do with data that comes from outside your program, like user input) you can never let your guard down and never disable the test, because servers can always go down, users can always give you bad data, files can always be corrupt. It is unsafe to disable these tests and so these should not be assertions. For a library function intended to be called by third-parties, the function arguments aren't under the control of the library author so should not be tested with assertions. But for an application where the author controls those function arguments, they are under the author's control and may be assertions or contracts. Design By Contract is partly a methodology and partly a set of syntax. Its a way of thinking about the design of your program. In practice, you don't have to buy 100% into DBC to get some benefit for it. A study done a few years back looked at 21 large projects in Eiffel, JML (Java) and Code Contracts for C# and found that 33% of the classes used contracts. http://se.ethz.ch/~meyer/publications/methodology/contract_analysis.pdf Like unit testing, you don't need 100% coverage to get benefit. 10% is better than nothing, and 20% is better than 10%. I wrote more about assertions here: https://import-that.dreamwidth.org/676.html -- Steve
Thanks for the explanation. This may be a bit OT, but is there a good way to do runtime assertions (in particular for data validation) that's as easy as assert? - self.assertEqual (unittest.TestCase.assertEqual) is hardly usable in other class instances, - pytest can be used at runtime but has somewhat nontrivial overhead - I always end up writing e.g. _assert() and _assertEqual() that throw AssertionErrors and helpful error messages just like unittest. How do contracts differ from checking interfaces with e.g. zope.interface? https://zopeinterface.readthedocs.io/en/latest/verify.html I'll read up a bit more on design by contract. I seem to have conceptualized dbc as a composition approach with typed interfaces and type and value assertions, but that's not it? If the parameter value assertions in pycontracts aren't really contracts, and the mypy compile-time parameter and return type checks are not really contracts, and zope.interface verifications aren't really contracts; what does that leave in terms of abstract data types, preconditions, postconditions, and invariants? https://en.wikipedia.org/wiki/Design_by_contract On Monday, August 27, 2018, Steven D'Aprano <steve@pearwood.info > wrote:
Hi, To clarify the benefits of the contracts, let me give you an example from our code base: @icontract.pre(lambda x: x >= 0) @icontract.pre(lambda y: y >= 0) @icontract.pre(lambda width: width >= 0) @icontract.pre(lambda height: height >= 0) @icontract.pre(lambda x, width, img: x + width <= pqry.opencv.width_of(img)) @icontract.pre(lambda y, height, img: y + height <= pqry.opencv.height_of(img)) @icontract.post(lambda self: (self.x, self.y) in self) @icontract.post(lambda self: (self.x + self.width - 1, self.y + self.height - 1) in self) @icontract.post(lambda self: (self.x + self.width, self.y + self.height) not in self) def __init__(self, img: np.ndarray, x: int, y: int, width: int, height: int) -> None: self.img = img[y:y + height, x:x + width].copy() self.x = x self.y = y self.width = width self.height = height def __contains__(self, xy: Tuple[int, int]) -> bool: x, y = xy return self.x <= x < self.x + self.width and \ self.y <= y < self.y + self.height We use mypy and type annotations for typing, and we don't need contracts for that. In this particular case, contracts are very handy to formulate how we deal with pixels. If you carefully look at the contracts, you will see: * pixels are indexed starting from 0 (as oppose to starting from 1) * region-of-interest needs to fit within an image. It can not be outside of its boundaries * pixels within the region-of-interest are in [x, x + width), [y, y + height) (where "[" means inclusive and ")" exclusive) *Why not types?* These constraints are either impossible or very hard to write as types. Moreover, I doubt that you could give such types meaningful names, and I expect the readability to suffer immensely. I suppose any constraints involving more than a couple of variables is hard to write as a type. *Benefits.* You could write all of this in human-readable informal docstring of a class, but then it would never be verified. This way we are sure that contracts are always there. Whenever we test, we can be sure that all the contracted properties hold. Mind that we do not test that they hold for that single test case -- we automatically test that they hold for all the test cases. This is, in my opinion, far superior to ambiguous human documentation or print statements (which are deleted after the debugging). I suppose *any* larger code base built by a team of more than one developer needs this kind of rigor. The contracts are not meant to be used only in high-risk applications -- they are meant to improve any code base. *Problem without standard support. *The current libraries (dpcontracts, icontract) can deal with pre and postconditions and class invariants of a concrete class. However, the current libraries break as soon as you have inheritance. There is no way in python to inherit the function decorators and to modify them. If we are to inherit from the above-mentioned class ROI, we need to make sure that the invariants of the parent class hold (*e.g., *what is contained in a ROI) as well as invariants of the child class. We might want to weaken the requirements (*e.g., *a ROI that can deal with pixels outside of an image) such that x and y can be an arbitrary numbers, not restricted to 0 <= x < img.width and 0 <= y < img.height respectively. Additionally, without standard approach, we do not know how to deal with contracts when we have a third-party tool that would like to use them (*e.g., *for automatic generation of unit tests, static testing or visualization in IDE or in documentation). @Wes Turner <wes.turner@gmail.com>: If I understood you correctly, you are looking for a library that gives you verbose messages when a contract is breached. Please have a look at icontract library: https://github.com/Parquery/icontract We added informative messages particularly because we wanted to have verbose output when something goes wrong.This was helpful not only during the development, but also in production since failure cases were hard to reproduce and anticipate in the first place (otherwise, they wouldn't have made it into the production). Here is an example with an error message:
On Tue, 28 Aug 2018 at 03:19, Wes Turner <wes.turner@gmail.com> wrote:
I now fully understand your use case. IIUC, what would make this easier is a syntax for defining preconditions, post conditions, and invariants that supports: * inheritance, composition (mixins) This isn't possible with decorators because there's no reference to the post-decorated function/method. A conventional method naming scheme and composition like _pre_check_[004]_height() would be unreasonably verbose; though it would then be possible to 'relax' specific contracts by name. * A better flow? Decorators subvert the normal top-down source layout (and so require up-down eye movements and working memory to remember what's going on) * Test case generation Generate tests and fixtures to verify the preconditions, postconditions, and invariants. * An Interpretor/compilation flag to skip the contracts/tests/assertions PYTHONOPTIMIZE (-O, -OO), __debug__ Isn't there now an -X <...> interpretor flag in CPython? On Tuesday, August 28, 2018, Marko Ristin-Kaufmann <marko.ristin@gmail.com> wrote:
I admit that I don't care that much about pre- and post-conditions. Like a lot of people I've written a you library to handle some cases, but was never motivates to make it better or use ones folks have refined. I definitely oppose dedicated syntax. Nonetheless, I'll say that it's really not hard to get a decorator to cooperate with inheritance if you want that. If you decorate a parent class you can simply attach the collection of invariants to the class as notations, along with whatever actual guards the decorator enforces. An inheriting child (also decorated) can just look up those notations and apply them to the child. I.e. the first thing a decorator can do is lookup the invariants attached to the parent (if any) and apply them to the child (if this behavior is enabled). Easy peasy. On Tue, Aug 28, 2018, 12:35 PM Wes Turner <wes.turner@gmail.com> wrote:
On Tue, Aug 28, 2018 at 07:46:02AM +0200, Marko Ristin-Kaufmann wrote:
Thanks for your example Marko. I think that is quite close to the ugliest piece of Python code I've ever seen, and I don't mean that as a criticism of you for writing it or the icontract library's design. We write code that our language allows us to write, not the code we want to write. Python has not been designed for contracts, and it shows. I think this demonstrates a truth about programming languages: Aethestics matter. Syntax matters. Some features, no matter how useful and beneficial, have no hope at all of getting widespread use if the syntax is awkward or the result ugly. I think that contracts will be doomed to be a minor niche used only by really keen proponents of the style so long as the best way to write a contract is to use a sequence of decorator calls with lambdas. I'm going to re-write that in a pseudo-Eiffel like syntax: def __init__(self, img: np.ndarray, x: int, y: int, width: int, height: int) -> None: require: x >= 0 y >= 0 width >= 0 height >= 0 x + width <= pqry.opencv.width_of(img) y + height <= pqry.opencv.height_of(img) # possibly this should be an invariant, not a post-condition? ensure: (self.x, self.y) in self (self.x + self.width - 1, self.y + self.height - 1) in self (self.x + self.width, self.y + self.height) not in self # body of __init__ goes here... This makes a huge difference to readability: - block structure to give visual shape to the code; - related code stays together (all the pre-conditions naturally go into the require block, post-conditions in the ensure block, and no chance of interleaving them @pre(...) @post(...) @pre(...) - no extraneous lambdas (they are pure noise); - no long sequence of decorators (more than one of each is pure noise); - possibility to write assertions which take more than one statement. -- Steve
How are conditions relaxed/overridden in Eiffel without a named reference? That looks much more readable. Expressions within such blocks are implicitly assertTrue(s). What sort of test generation from said nameless expressions would be most helpful? On Tuesday, August 28, 2018, Steven D'Aprano <steve@pearwood.info> wrote:
Wes Turner wrote:
I'm going to re-write that in a pseudo-Eiffel like syntax:
Maybe some magic could be done to make this work: def __init__(self, img: np.ndarray, x: int, y: int, width: int, height: int) -> None: def __require__(): x >= 0 y >= 0 width >= 0 height >= 0 x + width <= pqry.opencv.width_of(img) y + height <= pqry.opencv.height_of(img) def __ensure__(): (self.x, self.y) in self (self.x + self.width - 1, self.y + self.height - 1) in self (self.x + self.width, self.y + self.height) not in self # body of __init__ goes here... -- Greg
I've never used contracts, so excuse me if I didn't get how they would work, and what they should do. The last example is about pre-post conditions in a class constructor. But I imagine this is not the only case where one would want to define contracts, like probably: within methods that return something and to functions (and in both cases, we'd want to contractually state some of the return's specificities). Is the following function something someone used to work with contracts would write? def calculate(first: int, second: int) -> float: def __require__(): first > second second > 0 # or first > second > 0 ? def __ensure__(ret): # we automatically pass the return of the function to this one ret > 1 return first / second If so, having a reference to the function's output would probably be needed, as in the example above. Also, wouldn't someone who use contracts want the type hints he provided to be ensured without having to add requirements like `type(first) is int` or something? - Brice Le 29/08/2018 à 07:52, Greg Ewing a écrit :
On Wed, Aug 29, 2018 at 05:52:46PM +1200, Greg Ewing wrote:
The problem with this idea is that methods and functions are not declarations, but executable code. This __require__ function doesn't exist except while the __init__ method is running. So it can't be called before the __init__, it can't be called *automatically* (you need to call it yourself, from inside the __init__), and it can't be inherited. Of course with sufficient compiler magic of course the compiler could special case these methods and do whatever we want, but that seems like it would be just as much work but much uglier than using dedicated syntax. -- Steve
Op wo 29 aug. 2018 om 03:59 schreef Steven D'Aprano <steve@pearwood.info>:
What, really ? Well, it clearly shows you teach python and don't look much at code written by people who taught themselves. I taught myself, and the first .py file I created was over a 1000 lines, and contained the GUI in a 4-deep nested global dictionary, since I'd never seen a style guide at that point. (I got better)
On Wed, Aug 29, 2018 at 09:23:02AM +0200, Jacco van Dorp wrote:
[snip long sequence of @decorator(lambda) calls]
I said *close* :-)
Well, it clearly shows you teach python and don't look much at code written by people who taught themselves.
I didn't mean to compare it to code written by beginners. I meant professional quality. And I didn't mean it was *bad* code. Python is a remarkable elegant and pretty language, but there are some things that aren't a good fit to the existing syntax. Contracts are one. We can't easily write code in a declarative style like Prolog: sibling(X, Y) :- parent_child(Z, X), parent_child(Z, Y) (X is a sibling of Y when there exists some Z who is a parent of X and the same Z is the parent of Y); we have to re-write it in a procedural style. Some programming styles aren't a natural fit to a given syntax.
So far, nothing you describe is *necessarily* ugly or bad code. The std lib contains at least one file well over 1000 lines, and while it is complex code, its not ugly code by any means. And I couldn't judge the elegance of the dict unless I saw it and the alternatives :-) -- Steve
On Mon, Aug 27, 2018 at 09:04:21AM +0200, Jacco van Dorp wrote:
Total noob speaking here, but....
Those contracts are mostly important during development right ?
That assumes that production code is free of bugs. Usual practice in the Eiffel world is, I believe, to disable post- condition checking and other assertions in production code, but leave pre-condition checks in place. Note: for those unfamiliar with Design By Contract, pre- and post- condition checks and assertions are NOT used to detect predictable error states, such as bad user input or IO errors. They are only used for testing program correctness. A failed contract or assertion can only mean a bug in the program. In a totally bug-free program contracts can safely be disabled.
Slowdown isn't that much of an issue during development.
For many cases, it isn't an issue during production either.
In Eiffel, contracts can be disabled or enable globally, or on a class-by-class basis.
The differences between design by contract and sprinkling messages in the log include: - the compiler (or interpreter) never forgets to read the logs looking for failures; - when a failure occurs, the program halts, so you know its broken, rather than silently continuing and perhaps doing something unexpected; - you don't need to read the logs and try to remember why it is that you're printing the value of ``foo`` and whether its a good thing or a bad thing that it has the value you're seeing. Here's an introduction to design by contract: https://www.eiffel.org/doc/eiffel/ET-_Design_by_Contract_%28tm%29%2C_Asserti... -- Steve
On Thu, Aug 16, 2018 at 4:19 PM Elazar <elazarg@gmail.com> wrote:
You might also be interested in pep-563. although it is not intended for design by contract, it can help (syntactically).
FYI, PEP 563 doesn't help it. Read this section: https://www.python.org/dev/peps/pep-0563/#non-typing-usage-of-annotations "With this in mind, uses for annotations incompatible with the aforementioned PEPs should be considered deprecated." -- INADA Naoki <songofacandy@gmail.com>
Isn't the purpose of "assert" to be able to do design by contract ? assert test, "error message is the test fail" I mean, you just write your test, dev get a feedback on problems, and prod can remove all assert using -o. What more do you need ? Le 15/08/2018 à 23:06, Marko Ristin-Kaufmann a écrit :
Michel Desmoulin wrote:
Good question. My opinion is that assert statements are good. I like them. But wait, more is possible. Here are some ideas. 1. Checking the return value (or exception). This is a post-condition. 2. Checking return value, knowing the input values. This is a more sophisticated post-condition. 3. Adding checks around an untrusted function - possibly third party, possibly written in C. 4. Selective turning on and off of checking. The last two, selective checks around untrusted functions, I find particularly interesting. Suppose you have a solid, trusted, well-tested and reliable system. And you add, or change, a function called wibble(). In this situation, errors are most likely to be in wibble(), or in the interface to wibble(). So which checks are most valuable? I suggest the answer is 1. Checks internal to wibble. 2. Pre-conditions and post-conditions for wibble 3. Pre-conditions for any function called by wibble. Suppose wibble calls wobble. We should certainly have the system check wobble's preconditions, in this situation. But we don't need wobble to run checks all the time. Only when the immediate caller is wibble. I think assertions and design-by-contract point in similar directions. But design-by-contract takes you further, and is I suspect more valuable when the system being built is large. Thank you, Michel, for your good question. -- Jonathan
Hi, I implemented the inheritance via meta classes and function and class attributes for pre/postconditions and invariants, respectively. Unless I missed something, this is as far as we can go without the proper language support with a library based on decorators: https://github.com/Parquery/icontract (version 1.5.0) Note that it is actually a complete implementation of design-by-contract that supports both weakening of the preconditions and strengthening of the postconditions and invariants. Could you please have a look and let me know what you think about the current implementation? Once we are sure that there is nothing obvious missing, I'd like to move forward and discuss whether we could add this library (or rewrite it) into the standard Python libraries and what needs to be all fixed till to make it that far. Cheers, Marko On Sat, 8 Sep 2018 at 21:34, Jonathan Fine <jfine2358@gmail.com> wrote:
Hi, A brief follow-up (latest version 1.5.3): I removed the dependency on meta package so that now all comprehensions and generator expressions work. I still had to depend on asttokens in order to get the source code of the condition function. Is there maybe an alternative solution which uses only standard libraries? Any thoughts or feedback on the icontract library in general? Cheers, Marko On Mon, 10 Sep 2018 at 09:29, Marko Ristin-Kaufmann <marko.ristin@gmail.com> wrote:
Hi, Let me make a couple of practical examples from the work-in-progress ( https://github.com/Parquery/pypackagery, branch mristin/initial-version) to illustrate again the usefulness of the contracts and why they are, in my opinion, superior to assertions and unit tests. What follows is a list of function signatures decorated with contracts from pypackagery library preceded by a human-readable description of the contracts. The invariants tell us what format to expect from the related string properties. @icontract.inv(lambda self: self.name.strip() == self.name) @icontract.inv(lambda self: self.line.endswith("\n")) class Requirement: """Represent a requirement in requirements.txt.""" def __init__(self, name: str, line: str) -> None: """ Initialize. :param name: package name :param line: line in the requirements.txt file """ ... The postcondition tells us that the resulting map keys the values on their name property. @icontract.post(lambda result: all(val.name == key for key, val in result.items())) def parse_requirements(text: str, filename: str = '<unknown>') -> Mapping[str, Requirement]: """ Parse requirements file and return package name -> package requirement as in requirements.txt :param text: content of the ``requirements.txt`` :param filename: where we got the ``requirements.txt`` from (URL or path) :return: name of the requirement (*i.e.* pip package) -> parsed requirement """ ... The postcondition ensures that the resulting list contains only unique elements. Mind that if you returned a set, the order would have been lost. @icontract.post(lambda result: len(result) == len(set(result)), enabled=icontract.SLOW) def missing_requirements(module_to_requirement: Mapping[str, str], requirements: Mapping[str, Requirement]) -> List[str]: """ List requirements from module_to_requirement missing in the ``requirements``. :param module_to_requirement: parsed ``module_to_requiremnt.tsv`` :param requirements: parsed ``requirements.txt`` :return: list of requirement names """ ... Here is a bit more complex example. - The precondition A requires that all the supplied relative paths (rel_paths) are indeed relative (as opposed to absolute). - The postcondition B ensures that the initial set of paths (given in rel_paths) is included in the results. - The postcondition C ensures that the requirements in the results are the subset of the given requirements. - The precondition D requires that there are no missing requirements (*i.e. *that each requirement in the given module_to_requirement is also defined in the given requirements). @icontract.pre(lambda rel_paths: all(rel_pth.root == "" for rel_pth in rel_paths)) # A @icontract.post( lambda rel_paths, result: all(pth in result.rel_paths for pth in rel_paths), enabled=icontract.SLOW, description="Initial relative paths included") # B @icontract.post( lambda requirements, result: all(req.name in requirements for req in result.requirements), enabled=icontract.SLOW) # C @icontract.pre( lambda requirements, module_to_requirement: missing_requirements(module_to_requirement, requirements) == [], enabled=icontract.SLOW) # D def collect_dependency_graph(root_dir: pathlib.Path, rel_paths: List[pathlib.Path], requirements: Mapping[str, Requirement], module_to_requirement: Mapping[str, str]) -> Package: """ Collect the dependency graph of the initial set of python files from the code base. :param root_dir: root directory of the codebase such as "/home/marko/workspace/pqry/production/src/py" :param rel_paths: initial set of python files that we want to package. These paths are relative to root_dir. :param requirements: requirements of the whole code base, mapped by package name :param module_to_requirement: module to requirement correspondence of the whole code base :return: resolved depedendency graph including the given initial relative paths, """ I hope these examples convince you (at least a little bit :-)) that contracts are easier and clearer to write than asserts. As noted before in this thread, you can have the same *behavior* with asserts as long as you don't need to inherit the contracts. But the contract decorators make it very explicit what conditions should hold *without* having to look into the implementation. Moreover, it is very hard to ensure the postconditions with asserts as soon as you have a complex control flow since you would need to duplicate the assert at every return statement. (You could implement a context manager that ensures the postconditions, but a context manager is not more readable than decorators and you have to duplicate them as documentation in the docstring). In my view, contracts are also superior to many kinds of tests. As the contracts are *always* enforced, they also enforce the correctness throughout the program execution whereas the unit tests and doctests only cover a list of selected cases. Furthermore, writing the contracts in these examples as doctests or unit tests would escape the attention of most less experienced programmers which are not used to read unit tests as documentation. Finally, these unit tests would be much harder to read than the decorators (*e.g.*, the unit test would supply invalid arguments and then check for ValueError which is already a much more convoluted piece of code than the preconditions and postconditions as decorators. Such testing code also lives in a file separate from the original implementation making it much harder to locate and maintain). Mind that the contracts *do not* *replace* the unit tests or the doctests. The contracts make merely tests obsolete that test that the function or class actually observes the contracts. Design-by-contract helps you skip those tests and focus on the more complex ones that test the behavior. Another positive effect of the contracts is that they make your tests deeper: if you specified the contracts throughout the code base, a test of a function that calls other functions in its implementation will also make sure that all the contracts of that other functions hold. This can be difficult to implement with standard unit test frameworks. Another aspect of the design-by-contract, which is IMO ignored quite often, is the educational one. Contracts force the programmer to actually sit down and think *formally* about the inputs and the outputs (hopefully?) *before* she starts to implement a function. Since many schools use Python to teach programming (especially at high school level), I imagine writing contracts of a function to be a very good exercise in formal thinking for the students. Please let me know what points *do not *convince you that Python needs contracts (in whatever form -- be it as a standard library, be it as a language construct, be it as a widely adopted and collectively maintained third-party library). I would be very glad to address these points in my next message(s). Cheers, Marko
I'm afraid that in reading the examples provided it is difficulties for me not simply to think that EVERY SINGLE ONE of them would be FAR easier to read if it were an `assert` instead. The API of the library is a bit noisy, but I think the obstacle it's more in the higher level design for me. Adding many layers of expensive runtime checks and many lines of code in order to assure simple predicates that a glance at the code or unit tests would do better seems wasteful. I just cannot imagine wanting to write or work on the kind of codebase that is down here. If some people or organizations want to come in this manner, sure a library is great. But I definitely don't want it in the syntax, nor even in the standard library. On Sat, Sep 15, 2018, 2:53 AM Marko Ristin-Kaufmann <marko.ristin@gmail.com> wrote:
On Sep 15, 2018 2:53 AM, "Marko Ristin-Kaufmann" <marko.ristin@gmail.com> wrote: Hi, Let me make a couple of practical examples from the work-in-progress ( https://github.com/Parquery/pypackagery, branch mristin/initial-version) to illustrate again the usefulness of the contracts and why they are, in my opinion, superior to assertions and unit tests. What follows is a list of function signatures decorated with contracts from pypackagery library preceded by a human-readable description of the contracts. The invariants tell us what format to expect from the related string properties. @icontract.inv(lambda self: self.name.strip() == self.name) @icontract.inv(lambda self: self.line.endswith("\n")) class Requirement: """Represent a requirement in requirements.txt.""" def __init__(self, name: str, line: str) -> None: """ Initialize. :param name: package name :param line: line in the requirements.txt file """ ... The postcondition tells us that the resulting map keys the values on their name property. @icontract.post(lambda result: all(val.name == key for key, val in result.items())) def parse_requirements(text: str, filename: str = '<unknown>') -> Mapping[str, Requirement]: """ Parse requirements file and return package name -> package requirement as in requirements.txt :param text: content of the ``requirements.txt`` :param filename: where we got the ``requirements.txt`` from (URL or path) :return: name of the requirement (*i.e.* pip package) -> parsed requirement """ ... The postcondition ensures that the resulting list contains only unique elements. Mind that if you returned a set, the order would have been lost. @icontract.post(lambda result: len(result) == len(set(result)), enabled=icontract.SLOW) def missing_requirements(module_to_requirement: Mapping[str, str], requirements: Mapping[str, Requirement]) -> List[str]: """ List requirements from module_to_requirement missing in the ``requirements``. :param module_to_requirement: parsed ``module_to_requiremnt.tsv`` :param requirements: parsed ``requirements.txt`` :return: list of requirement names """ ... Here is a bit more complex example. - The precondition A requires that all the supplied relative paths (rel_paths) are indeed relative (as opposed to absolute). - The postcondition B ensures that the initial set of paths (given in rel_paths) is included in the results. - The postcondition C ensures that the requirements in the results are the subset of the given requirements. - The precondition D requires that there are no missing requirements (*i.e. *that each requirement in the given module_to_requirement is also defined in the given requirements). @icontract.pre(lambda rel_paths: all(rel_pth.root == "" for rel_pth in rel_paths)) # A @icontract.post( lambda rel_paths, result: all(pth in result.rel_paths for pth in rel_paths), enabled=icontract.SLOW, description="Initial relative paths included") # B @icontract.post( lambda requirements, result: all(req.name in requirements for req in result.requirements), enabled=icontract.SLOW) # C @icontract.pre( lambda requirements, module_to_requirement: missing_requirements(module_to_requirement, requirements) == [], enabled=icontract.SLOW) # D def collect_dependency_graph(root_dir: pathlib.Path, rel_paths: List[pathlib.Path], requirements: Mapping[str, Requirement], module_to_requirement: Mapping[str, str]) -> Package: """ Collect the dependency graph of the initial set of python files from the code base. :param root_dir: root directory of the codebase such as "/home/marko/workspace/pqry/production/src/py" :param rel_paths: initial set of python files that we want to package. These paths are relative to root_dir. :param requirements: requirements of the whole code base, mapped by package name :param module_to_requirement: module to requirement correspondence of the whole code base :return: resolved depedendency graph including the given initial relative paths, """ I hope these examples convince you (at least a little bit :-)) that contracts are easier and clearer to write than asserts. As noted before in this thread, you can have the same *behavior* with asserts as long as you don't need to inherit the contracts. But the contract decorators make it very explicit what conditions should hold *without* having to look into the implementation. Moreover, it is very hard to ensure the postconditions with asserts as soon as you have a complex control flow since you would need to duplicate the assert at every return statement. (You could implement a context manager that ensures the postconditions, but a context manager is not more readable than decorators and you have to duplicate them as documentation in the docstring). In my view, contracts are also superior to many kinds of tests. As the contracts are *always* enforced, they also enforce the correctness throughout the program execution whereas the unit tests and doctests only cover a list of selected cases. Furthermore, writing the contracts in these examples as doctests or unit tests would escape the attention of most less experienced programmers which are not used to read unit tests as documentation. Finally, these unit tests would be much harder to read than the decorators (*e.g.*, the unit test would supply invalid arguments and then check for ValueError which is already a much more convoluted piece of code than the preconditions and postconditions as decorators. Such testing code also lives in a file separate from the original implementation making it much harder to locate and maintain). Mind that the contracts *do not* *replace* the unit tests or the doctests. The contracts make merely tests obsolete that test that the function or class actually observes the contracts. Design-by-contract helps you skip those tests and focus on the more complex ones that test the behavior. Another positive effect of the contracts is that they make your tests deeper: if you specified the contracts throughout the code base, a test of a function that calls other functions in its implementation will also make sure that all the contracts of that other functions hold. This can be difficult to implement with standard unit test frameworks. Another aspect of the design-by-contract, which is IMO ignored quite often, is the educational one. Contracts force the programmer to actually sit down and think *formally* about the inputs and the outputs (hopefully?) *before* she starts to implement a function. Since many schools use Python to teach programming (especially at high school level), I imagine writing contracts of a function to be a very good exercise in formal thinking for the students. Please let me know what points *do not *convince you that Python needs contracts (in whatever form -- be it as a standard library, be it as a language construct, be it as a widely adopted and collectively maintained third-party library). I would be very glad to address these points in my next message(s). Cheers, Marko _______________________________________________ Python-ideas mailing list Python-ideas@python.org https://mail.python.org/mailman/listinfo/python-ideas Code of Conduct: http://python.org/psf/codeofconduct/
I just want to point out that you don't need permission from anybody to start a library. I think developing and popularizing a contracts library is a reasonable goal -- but that's something you can start doing at any time without waiting for consensus. And if it gets popular enough, maybe it'll be added to the standard library in some form. That's what happened with attrs, iirc -- it got fairly popular and demonstrated there was an unfilled niche, and so Python acquired dataclasses.. The contracts make merely tests obsolete that test that the function or
class actually observes the contracts.
Is this actually the case? Your contracts are only checked when the function is evaluated, so you'd still need to write that unit test that confirms the function actually observes the contract. I don't think you necessarily get to reduce the number of tests you'd need to write. Please let me know what points *do not *convince you that Python needs
contracts
While I agree that contracts are a useful tool, I don't think they're going to be necessarily useful for *all* Python programmers. For example, contracts aren't particularly useful if you're writing fairly straightforward code with relatively simple invariants. I'm also not convinced that libraries where contracts are checked specifically *at runtime* actually give you that much added power and impact. For example, you still need to write a decent number of unit tests to make sure your contracts are being upheld (unless you plan on checking this by just deploying your code and letting it run, which seems suboptimal). There's also no guarantee that your contracts will necessarily be *accurate*. It's entirely possible that your preconditions/postconditions might hold for every test case you can think of, but end up failing when running in production due to some edge case that you missed. (And if you decide to disable those pre/post conditions to avoid the efficiency hit, you're back to square zero.) Or I guess to put it another way -- it seems what all of these contract libraries are doing is basically adding syntax to try and make adding asserts in various places more ergonomic, and not much else. I agree those kinds of libraries can be useful, but I don't think they're necessarily useful enough to be part of the standard library or to be a technique Python programmers should automatically use by default. What might be interesting is somebody wrote a library that does something more then just adding asserts. For example, one idea might be to try hooking up a contracts library to hypothesis (or any other library that does quickcheck-style testing). That might be a good way of partially addressing the problems up above -- you write out your invariants, and a testing library extracts that information and uses it to automatically synthesize interesting test cases. (And of course, what would be very cool is if the contracts could be verified statically like you can do in languages like dafny -- that way, you genuinely would be able to avoid writing many kinds of tests and could have confidence your contracts are upheld. But I understanding implementing such verifiers are extremely challenging and would probably have too-steep of a learning curve to be usable by most people anyways.) -- Michael On Fri, Sep 14, 2018 at 11:51 PM, Marko Ristin-Kaufmann < marko.ristin@gmail.com> wrote:
Hi David Maertz and Michael Lee, Thank you for raising the points. Please let me respond to your comments in separation. Please let me know if I missed or misunderstood anything. *Assertions versus contracts.* David wrote:
I think there are two misunderstandings on the role of the contracts. First, they are part of the function signature, and not of the implementation. In contrast, the assertions are part of the implementation and are completely obscured in the signature. To see the contracts of a function or a class written as assertions, you need to visually inspect the implementation. The contracts are instead engraved in the signature and immediately visible. For example, you can test the distinction by pressing Ctrl + q in Pycharm. Second, assertions are only suitable for preconditions. Postconditions are practically unmaintainable as assertions as soon as you have multiple early returns in a function. The invariants implemented as assertions are always unmaintainable in practice (except for very, very small classes) -- you need to inspect each function of the class and all their return statements and manually add assertions for each invariant. Removing or changing invariants manually is totally impractical in my view. *Efficiency and Evidency. *David wrote:
I'm not very sure what you mean by expensive runtime checks -- every single contract can be disabled at any point. Once a contract is disabled, there is literally no runtime computational cost incurred. The complexity of a contract during testing is also exactly the same as if you wrote it in the unit test. There is a constant overhead due to the extra function call to check the condition, but there's no more time complexity to it. The overhead of an additional function call is negligible in most practical test cases. When you say "a glance at the code", this implies to me that you referring to your own code and not to legacy code. In my experience, even simple predicates are often not obvious to see in other people's code as one might think (*e.g. *I had to struggle with even most simple ones like whether the result ends in a newline or not -- often having to actually run the code to check experimentally what happens with different inputs). Postconditions prove very useful in such situations: they let us know that whenever a function returns, the result must satisfy its postconditions. They are formal and obvious to read in the function signature, and hence spare us the need to parse the function's implementation or run it. Contracts in the unit tests.
(emphasis mine) Defining contracts in a unit test is, as I already mentioned in my previous message, problematic due to two reasons. First, the contract resides in a place far away from the function definition which might make it hard to find and maintain. Second, defining the contract in the unit test makes it impossible to put the contract in the production or test it in a call from a different function. In contrast, introducing the contract as a decorator works perfectly fine in all the three above-mentioned cases (smoke unit test, production, deeper testing). *Library. *Michael wrote:
As a matter of fact, I already implemented the library which covers most of the design-by-contract including the inheritance of the contracts. (The only missing parts are retrieval of "old" values in postconditions and loop invariants.) It's published on pypi as "icontract" package (the website is https://github.com/Parquery/icontract/). I'd like to gauge the interest before I/we even try to make a proposal to make it into the standard library. The discussions in this thread are an immense help for me to crystallize the points that would need to be addressed explicitly in such a proposal. If the proposal never comes about, it would at least flow into the documentation of the library and help me identify and explain better the important points. *Observation of contracts. *Michael wrote:
Assuming that a contracts library is working correctly, there is no need to test whether a contract is observed or not -- you assume it is. The same applies to any testing library -- otherwise, you would have to test the tester, and so on *ad infinitum.* You still need to evaluate the function during testing, of course. But you don't need to document the contracts in your tests nor check that the postconditions are enforced -- you assume that they hold. For example, if you introduce a postcondition that the result of a function ends in a newline, there is no point of making a unit test, passing it some value and then checking that the result value ends in a newline in the test. Normally, it is sufficient to smoke-test the function. For example, you write a smoke unit test that gives a range of inputs to the function by using hypothesis library and let the postconditions be automatically checked. You can view each postcondition as an additional test case in this scenario -- but one that is also embedded in the function signature and also applicable in production. Not all tests can be written like this, of course. Dealing with a complex function involves writing testing logic which is too complex to fit in postconditions. Contracts are not a panacea, but they absolute us from implementing trivial testing logic while keeping the important bits of the documentation close to the function and allowing for deeper tests. *Accurate contracts. *Michael wrote:
Unfortunately, there is no practical exit from this dilemma -- and it applies all the same for the tests. Who guarantees that the testing logic of the unit tests are correct? Unless you can formally prove that the code does what it should, there is no way around it. Whether you write contracts in the tests or in the decorators, it makes no difference to accuracy. If you missed to test an edge case, well, you missed it :). The design-by-contract does not make the code bug-free, but makes the bugs *much less likely* and *easier *to detect *early*. In practice, if there is a complex contract, I encapsulate its complex parts in separate functions (often with their own contracts), test these functions in separation and then, once the tests pass and I'm confident about their correctness, put them into contracts. (And if you decide to disable those pre/post conditions to avoid the
efficiency hit, you're back to square zero.)
In practice, we at Parquery AG let the critical contracts to run in production to ensure that the program blows up before it exercises undefined behavior in a critical situation. The informative violation errors of the icontract library help us to trace the bugs more easily since the relevant values are part of the error log. However, if some of the contracts are too inefficient to check in production, alas you have to turn them off and they can't be checked since they are inefficient. This seems like a tautology to me -- could you please clarify a bit what you meant? If a check is critical and inefficient at the same time then your problem is unsolvable (or at least ill-defined); contracts as well as any other approach can not solve it. *Ergonimical assertions. *Michael wrote:
Please do not underestimate another aspect of the contracts, namely the value of contracts as verifiable documentation. Please note that the only alternative that I observe in practice without design-by-contract is to write contracts in docstrings in *natural language*. Most often, they are just assumed, so the next programmer burns her fingers expecting the contracts to hold when they actually differ from the class or function description, but nobody bothered to update the docstrings (which is a common pitfall in any code base over a longer period of time). *Automatic generation of tests.* Michael wrote:
This is the final goal and my main motivation to push for design-by-contract in Python :). There is a whole research community that tries to come up with automatic test generations, and contracts are of great utility there. Mind that generating the tests based on contracts is not trivial: hypothesis just picks elements for each input independently which is a much easier problem. However, preconditions can define how the arguments are *related*. Assume a function takes two numbers as arguments, x and y. If the precondition is y < x < (y + x) * 10, it is not trivial even for this simple example to come up with concrete samples of x and y unless you simply brute-force the problem by densely sampling all the numbers and checking the precondition. I see a chicken-and-egg problem here. If design-by-contract is not widely adopted, there will also be fewer or no libraries for automatic test generation. Honestly, I have absolutely no idea how you could approach automatic generation of test cases without contracts (in one form or the other). For example, how could you automatically mock a class without knowing its invariants? Since generating test cases for functions with non-trivial contracts is hard (and involves collaboration of many people), I don't expect anybody to start even thinking about it if the tool can only be applied to almost anywhere due to lack of contracts. Formal proofs and static analysis are even harder beasts to tame -- and I'd say the argument holds true for them even more. David and Michael, thank you again for your comments! I welcome very much your opinion and any follow-ups as well as from other participants on this mail list. Cheers, Marko On Sat, 15 Sep 2018 at 10:42, Michael Lee <michael.lee.0x2a@gmail.com> wrote:
Hi, Again a brief update. * icontract supports now static and class methods (thanks to my colleague Adam Radomski) which came very handy when defining a group of functions as an interface *via* an abstract (stateless) class. The implementors then need to all satisfy the contracts without needing to re-write them. You could implement the same behavior with *_impl or _* ("protected") methods where public methods would add the contracts as asserts, but we find the contracts-as-decorators more elegant (N functions instead of 2*N; see the snippet below). * We implemented a linter to statically check that the contract arguments are defined correctly. It is available as a separate Pypi package pyicontract-lint (https://github.com/Parquery/pyicontract-lint/). Next step will be to use asteroid to infer that the return type of the condition function is boolean. Does it make sense to include PEX in the release on github? * We plan to implement a sphinx plugin so that contracts can be readily visible in the documentation. Is there any guideline or standard/preferred approach how you would expect this plugin to be implemented? My colleagues and I don't have any experience with sphinx plugins, so any guidance is very welcome. class Component(abc.ABC, icontract.DBC): """Initialize a single component.""" @staticmethod @abc.abstractmethod def user() -> str: """ Get the user name. :return: user which executes this component. """ pass @staticmethod @abc.abstractmethod @icontract.post(lambda result: result in groups()) def primary_group() -> str: """ Get the primary group. :return: primary group of this component """ pass @staticmethod @abc.abstractmethod @icontract.post(lambda result: result.issubset(groups())) def secondary_groups() -> Set[str]: """ Get the secondary groups. :return: list of secondary groups """ pass @staticmethod @abc.abstractmethod @icontract.post(lambda result: all(not pth.is_absolute() for pth in result)) def bin_paths(config: mapried.config.Config) -> List[pathlib.Path]: """ Get list of binary paths used by this component. :param config: of the instance :return: list of paths to binaries used by this component """ pass @staticmethod @abc.abstractmethod @icontract.post(lambda result: all(not pth.is_absolute() for pth in result)) def py_paths(config: mapried.config.Config) -> List[pathlib.Path]: """ Get list of py paths used by this component. :param config: of the instance :return: list of paths to python executables used by this component """ pass @staticmethod @abc.abstractmethod @icontract.post(lambda result: all(not pth.is_absolute() for pth in result)) def dirs(config: mapried.config.Config) -> List[pathlib.Path]: """ Get directories used by this component. :param config: of the instance :return: list of paths to directories used by this component """ pass On Sat, 15 Sep 2018 at 22:14, Marko Ristin-Kaufmann <marko.ristin@gmail.com> wrote:
Hi, I implemented a sphinx extension to include contracts in the documentation: https://github.com/Parquery/sphinx-icontract The extension supports inheritance. It lists all the postconditions and invariants including the inherited one. The preconditions are grouped by classes with ":requires:" and ":requires else:". I was unable to get the syntax highlighting for in-line code to work -- does anybody know how to do that in Sphinx? The results can be seen, *e.g.* in this documentation: https://pypackagery.readthedocs.io/en/latest/packagery.html On a more general note: is there any blocker left why you would *not *use the contracts in your code? Anything I could improve or fix in icontract that would make it more convincing to use (apart from implementing static contract checking and automatic test generation :))? Cheers, Marko On Thu, 20 Sep 2018 at 22:52, Marko Ristin-Kaufmann <marko.ristin@gmail.com> wrote:
participants (19)
-
Brice Parent -
Chris Angelico -
Daniel Moisset -
David Mertz -
Elazar -
Eric Fahlgren -
Eric V. Smith -
Greg Ewing -
INADA Naoki -
Ivan Levkivskyi -
Jacco van Dorp -
Jonathan Fine -
Marko Ristin-Kaufmann -
Michael Lee -
Michel Desmoulin -
Paul Moore -
Stephan Houben -
Steven D'Aprano -
Wes Turner