[Python-ideas] Pre-conditions and post-conditions
David Mertz
mertz at gnosis.cx
Sat Sep 15 04:29:11 EDT 2018
I'm afraid that in reading the examples provided it is difficulties for me
not simply to think that EVERY SINGLE ONE of them would be FAR easier to
read if it were an `assert` instead.
The API of the library is a bit noisy, but I think the obstacle it's more
in the higher level design for me. Adding many layers of expensive runtime
checks and many lines of code in order to assure simple predicates that a
glance at the code or unit tests would do better seems wasteful.
I just cannot imagine wanting to write or work on the kind of codebase that
is down here. If some people or organizations want to come in this manner,
sure a library is great. But I definitely don't want it in the syntax, nor
even in the standard library.
On Sat, Sep 15, 2018, 2:53 AM Marko Ristin-Kaufmann <marko.ristin at gmail.com>
wrote:
> Hi,
> Let me make a couple of practical examples from the work-in-progress (
> https://github.com/Parquery/pypackagery, branch mristin/initial-version)
> to illustrate again the usefulness of the contracts and why they are, in my
> opinion, superior to assertions and unit tests.
>
> What follows is a list of function signatures decorated with contracts
> from pypackagery library preceded by a human-readable description of the
> contracts.
>
> The invariants tell us what format to expect from the related string
> properties.
>
> @icontract.inv(lambda self: self.name.strip() == self.name)
> @icontract.inv(lambda self: self.line.endswith("\n"))
> class Requirement:
> """Represent a requirement in requirements.txt."""
>
> def __init__(self, name: str, line: str) -> None:
> """
> Initialize.
>
> :param name: package name
> :param line: line in the requirements.txt file
> """
> ...
>
> The postcondition tells us that the resulting map keys the values on their
> name property.
>
> @icontract.post(lambda result: all(val.name == key for key, val in result.items()))
> def parse_requirements(text: str, filename: str = '<unknown>') -> Mapping[str, Requirement]:
> """
> Parse requirements file and return package name -> package requirement as in requirements.txt
>
> :param text: content of the ``requirements.txt``
> :param filename: where we got the ``requirements.txt`` from (URL or path)
> :return: name of the requirement (*i.e.* pip package) -> parsed requirement
> """
> ...
>
>
> The postcondition ensures that the resulting list contains only unique
> elements. Mind that if you returned a set, the order would have been lost.
>
> @icontract.post(lambda result: len(result) == len(set(result)), enabled=icontract.SLOW)
> def missing_requirements(module_to_requirement: Mapping[str, str],
> requirements: Mapping[str, Requirement]) -> List[str]:
> """
> List requirements from module_to_requirement missing in the ``requirements``.
>
> :param module_to_requirement: parsed ``module_to_requiremnt.tsv``
> :param requirements: parsed ``requirements.txt``
> :return: list of requirement names
> """
> ...
>
> Here is a bit more complex example.
> - The precondition A requires that all the supplied relative paths
> (rel_paths) are indeed relative (as opposed to absolute).
> - The postcondition B ensures that the initial set of paths (given in
> rel_paths) is included in the results.
> - The postcondition C ensures that the requirements in the results are the
> subset of the given requirements.
> - The precondition D requires that there are no missing requirements (*i.e.
> *that each requirement in the given module_to_requirement is also defined
> in the given requirements).
>
> @icontract.pre(lambda rel_paths: all(rel_pth.root == "" for rel_pth in rel_paths)) # A
> @icontract.post(
> lambda rel_paths, result: all(pth in result.rel_paths for pth in rel_paths),
> enabled=icontract.SLOW,
> description="Initial relative paths included") # B
> @icontract.post(
> lambda requirements, result: all(req.name in requirements for req in result.requirements),
> enabled=icontract.SLOW) # C
> @icontract.pre(
> lambda requirements, module_to_requirement: missing_requirements(module_to_requirement, requirements) == [],
> enabled=icontract.SLOW) # D
> def collect_dependency_graph(root_dir: pathlib.Path, rel_paths: List[pathlib.Path],
> requirements: Mapping[str, Requirement],
> module_to_requirement: Mapping[str, str]) -> Package:
>
> """
> Collect the dependency graph of the initial set of python files from the code base.
>
> :param root_dir: root directory of the codebase such as "/home/marko/workspace/pqry/production/src/py"
> :param rel_paths: initial set of python files that we want to package. These paths are relative to root_dir.
> :param requirements: requirements of the whole code base, mapped by package name
> :param module_to_requirement: module to requirement correspondence of the whole code base
> :return: resolved depedendency graph including the given initial relative paths,
> """
>
> I hope these examples convince you (at least a little bit :-)) that
> contracts are easier and clearer to write than asserts. As noted before in
> this thread, you can have the same *behavior* with asserts as long as you
> don't need to inherit the contracts. But the contract decorators make it
> very explicit what conditions should hold *without* having to look into
> the implementation. Moreover, it is very hard to ensure the postconditions
> with asserts as soon as you have a complex control flow since you would
> need to duplicate the assert at every return statement. (You could
> implement a context manager that ensures the postconditions, but a context
> manager is not more readable than decorators and you have to duplicate them
> as documentation in the docstring).
>
> In my view, contracts are also superior to many kinds of tests. As the
> contracts are *always* enforced, they also enforce the correctness
> throughout the program execution whereas the unit tests and doctests only
> cover a list of selected cases. Furthermore, writing the contracts in these
> examples as doctests or unit tests would escape the attention of most less
> experienced programmers which are not used to read unit tests as
> documentation. Finally, these unit tests would be much harder to read than
> the decorators (*e.g.*, the unit test would supply invalid arguments and
> then check for ValueError which is already a much more convoluted piece of
> code than the preconditions and postconditions as decorators. Such testing
> code also lives in a file separate from the original implementation making
> it much harder to locate and maintain).
>
> Mind that the contracts *do not* *replace* the unit tests or the
> doctests. The contracts make merely tests obsolete that test that the
> function or class actually observes the contracts. Design-by-contract helps
> you skip those tests and focus on the more complex ones that test the
> behavior. Another positive effect of the contracts is that they make your
> tests deeper: if you specified the contracts throughout the code base, a
> test of a function that calls other functions in its implementation will
> also make sure that all the contracts of that other functions hold. This
> can be difficult to implement with standard unit test frameworks.
>
> Another aspect of the design-by-contract, which is IMO ignored quite
> often, is the educational one. Contracts force the programmer to actually
> sit down and think *formally* about the inputs and the outputs
> (hopefully?) *before* she starts to implement a function. Since many
> schools use Python to teach programming (especially at high school level),
> I imagine writing contracts of a function to be a very good exercise in
> formal thinking for the students.
>
> Please let me know what points *do not *convince you that Python needs
> contracts (in whatever form -- be it as a standard library, be it as a
> language construct, be it as a widely adopted and collectively maintained
> third-party library). I would be very glad to address these points in my
> next message(s).
>
> Cheers,
> Marko
> _______________________________________________
> Python-ideas mailing list
> Python-ideas at python.org
> https://mail.python.org/mailman/listinfo/python-ideas
> Code of Conduct: http://python.org/psf/codeofconduct/
>
On Sep 15, 2018 2:53 AM, "Marko Ristin-Kaufmann" <marko.ristin at gmail.com>
wrote:
Hi,
Let me make a couple of practical examples from the work-in-progress (
https://github.com/Parquery/pypackagery, branch mristin/initial-version) to
illustrate again the usefulness of the contracts and why they are, in my
opinion, superior to assertions and unit tests.
What follows is a list of function signatures decorated with contracts from
pypackagery library preceded by a human-readable description of the
contracts.
The invariants tell us what format to expect from the related string
properties.
@icontract.inv(lambda self: self.name.strip() == self.name)
@icontract.inv(lambda self: self.line.endswith("\n"))
class Requirement:
"""Represent a requirement in requirements.txt."""
def __init__(self, name: str, line: str) -> None:
"""
Initialize.
:param name: package name
:param line: line in the requirements.txt file
"""
...
The postcondition tells us that the resulting map keys the values on their
name property.
@icontract.post(lambda result: all(val.name == key for key, val in
result.items()))
def parse_requirements(text: str, filename: str = '<unknown>') ->
Mapping[str, Requirement]:
"""
Parse requirements file and return package name -> package
requirement as in requirements.txt
:param text: content of the ``requirements.txt``
:param filename: where we got the ``requirements.txt`` from (URL or path)
:return: name of the requirement (*i.e.* pip package) -> parsed requirement
"""
...
The postcondition ensures that the resulting list contains only unique
elements. Mind that if you returned a set, the order would have been lost.
@icontract.post(lambda result: len(result) == len(set(result)),
enabled=icontract.SLOW)
def missing_requirements(module_to_requirement: Mapping[str, str],
requirements: Mapping[str, Requirement]) -> List[str]:
"""
List requirements from module_to_requirement missing in the
``requirements``.
:param module_to_requirement: parsed ``module_to_requiremnt.tsv``
:param requirements: parsed ``requirements.txt``
:return: list of requirement names
"""
...
Here is a bit more complex example.
- The precondition A requires that all the supplied relative paths
(rel_paths) are indeed relative (as opposed to absolute).
- The postcondition B ensures that the initial set of paths (given in
rel_paths) is included in the results.
- The postcondition C ensures that the requirements in the results are the
subset of the given requirements.
- The precondition D requires that there are no missing requirements (*i.e.
*that each requirement in the given module_to_requirement is also defined
in the given requirements).
@icontract.pre(lambda rel_paths: all(rel_pth.root == "" for rel_pth in
rel_paths)) # A
@icontract.post(
lambda rel_paths, result: all(pth in result.rel_paths for pth in rel_paths),
enabled=icontract.SLOW,
description="Initial relative paths included") # B
@icontract.post(
lambda requirements, result: all(req.name in requirements for req
in result.requirements),
enabled=icontract.SLOW) # C
@icontract.pre(
lambda requirements, module_to_requirement:
missing_requirements(module_to_requirement, requirements) == [],
enabled=icontract.SLOW) # D
def collect_dependency_graph(root_dir: pathlib.Path, rel_paths:
List[pathlib.Path],
requirements: Mapping[str, Requirement],
module_to_requirement: Mapping[str, str])
-> Package:
"""
Collect the dependency graph of the initial set of python files
from the code base.
:param root_dir: root directory of the codebase such as
"/home/marko/workspace/pqry/production/src/py"
:param rel_paths: initial set of python files that we want to
package. These paths are relative to root_dir.
:param requirements: requirements of the whole code base, mapped
by package name
:param module_to_requirement: module to requirement correspondence
of the whole code base
:return: resolved depedendency graph including the given initial
relative paths,
"""
I hope these examples convince you (at least a little bit :-)) that
contracts are easier and clearer to write than asserts. As noted before in
this thread, you can have the same *behavior* with asserts as long as you
don't need to inherit the contracts. But the contract decorators make it
very explicit what conditions should hold *without* having to look into the
implementation. Moreover, it is very hard to ensure the postconditions with
asserts as soon as you have a complex control flow since you would need to
duplicate the assert at every return statement. (You could implement a
context manager that ensures the postconditions, but a context manager is
not more readable than decorators and you have to duplicate them as
documentation in the docstring).
In my view, contracts are also superior to many kinds of tests. As the
contracts are *always* enforced, they also enforce the correctness
throughout the program execution whereas the unit tests and doctests only
cover a list of selected cases. Furthermore, writing the contracts in these
examples as doctests or unit tests would escape the attention of most less
experienced programmers which are not used to read unit tests as
documentation. Finally, these unit tests would be much harder to read than
the decorators (*e.g.*, the unit test would supply invalid arguments and
then check for ValueError which is already a much more convoluted piece of
code than the preconditions and postconditions as decorators. Such testing
code also lives in a file separate from the original implementation making
it much harder to locate and maintain).
Mind that the contracts *do not* *replace* the unit tests or the doctests.
The contracts make merely tests obsolete that test that the function or
class actually observes the contracts. Design-by-contract helps you skip
those tests and focus on the more complex ones that test the behavior.
Another positive effect of the contracts is that they make your tests
deeper: if you specified the contracts throughout the code base, a test of
a function that calls other functions in its implementation will also make
sure that all the contracts of that other functions hold. This can be
difficult to implement with standard unit test frameworks.
Another aspect of the design-by-contract, which is IMO ignored quite often,
is the educational one. Contracts force the programmer to actually sit down
and think *formally* about the inputs and the outputs (hopefully?) *before*
she starts to implement a function. Since many schools use Python to teach
programming (especially at high school level), I imagine writing contracts
of a function to be a very good exercise in formal thinking for the
students.
Please let me know what points *do not *convince you that Python needs
contracts (in whatever form -- be it as a standard library, be it as a
language construct, be it as a widely adopted and collectively maintained
third-party library). I would be very glad to address these points in my
next message(s).
Cheers,
Marko
_______________________________________________
Python-ideas mailing list
Python-ideas at python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20180915/b27e3a43/attachment-0001.html>
More information about the Python-ideas
mailing list