On Sat, Sep 26, 2020 at 8:50 AM Sascha Schlemmer via Python-ideas < python-ideas@python.org> wrote:

I think a clear distinction has to be made between errors that belong to the api surface of a function e.g. some `ValueError` or `ZeroDivisionError` and *exceptional* errors e,g.`KeyboardInterrupt` or `DatabaseExplodedError`. For the latter case exceptions work perfectly well as they are and it is both unhelpful and infeasible to annotate/document every exception - just let the exception bubble up and someone will handle it (might even be a process supervisor like supervisord)

For the first case I think a better way to model a function, that has some *expected* failure mode is to try and write a *total* function that doesn’t use exceptions at all. Let me show what I mean:

``` def div(a: int, b: int) -> float: return a / b ```

This function tries to divide two integers, trouble is that `div` is not defined for every input we may pass it, i.e. passing `b=0` will lead to `ZeroDivisionError` being raised. In other words `div` is a *partial* function because it is not defined for every member of its domain (type of its arguments).

There are two ways to amend this issue and make `div` a *total* function that is honest about its domain and co-domain (types of input arguments and return type):

1. Extending the type of the functions co-domain (return type) and making the error case explicit (see https://returns.readthedocs.io/en/latest/index.html for a library that implements this style of error handling)

``` def div(a: int, b: int) -> Maybe[float]: try: return Some(a / b) except ZeroDivisionError: return Nothing() ```

or

``` def div(a: int, b: int) -> Result[float, ZeroDivisionError]: try: return Success(a / b) except ZeroDivisionError as error: return Failure(error) ```

Now `div` does return a valid instance of `Maybe` (or `Result` if more a more detailed failure case is wanted) which is either something like `Some(3.1415) ` or `Nothing` (analogous to `None` ). The caller of the function then has to deal with this and mypy will correctly warn if the user fails to do so properly e.g.

´´´ div(1,1) + 1 # mypy will give a type error ´´´

https://en.wikipedia.org/wiki/Partial_function https://en.wikipedia.org/wiki/Partial_application#Implementations - https://docs.python.org/3/library/functools.html#functools.partial - https://docs.python.org/3/library/functools.html#functools.singledispatch PEP 0622: Structural Pattern Matching https://www.python.org/dev/peps/pep-0622/

2. Restricting the functions domain to values with defined behavior

``` def div(a: int, b: NonZeroInteger) -> float: return a / b ```

In this case the burden is put on the caller to supply valid inputs to the function and this effectively pushes the error handling/ input validation out of `div`

https://pypi.org/project/icontract/ : ```python

...

...

import icontract @icontract.require(lambda x: x > 3) ... def some_func(x: int, y: int = 5)->None: ... pass ... some_func(x=5) # Pre-condition violation some_func(x=1) Traceback (most recent call last): ... icontract.errors.ViolationError: File , line 1 in <module>: x > 3: x was 1

There exist a couple of contract libraries. However, at the time of this
> writing (September 2018), they all required the programmer either to learn
> a new syntax (PyContracts) or to write redundant condition descriptions (
> e.g., contracts, covenant, dpcontracts, pyadbc and pcd).


https://andreacensi.github.io/contracts/ supports runtime argument and
return constraints as Python 3 annotations, with the @contract decorator,
and in docstrings:

```python
@contract
def my_function(a : 'int,>0', b : 'list[N],N>0') -> 'list[N]':
     # Requires b to be a nonempty list, and the return
     # value to have the same length.
     ...

icontract supports inheritance. Defining the domain and range of a partial callable is relevant to this discussion about callables that don't return a value or None, per se, but instead raise Exceptions. Results looks neat. I think that just reading about all of these tools could make me a better programmer. What an ironic conflation of None with null void pointer references here (see also Pandas' handling of None/np.NaN) https://returns.readthedocs.io/en/latest/index.html#id1 :

Or you can use Maybe container! It consists of Some and Nothing types, representing existing state and empty (instead of None) state respectively.

A language that does all this really well is F# (which like python is a multi-paradigm language that offers both object-oriented and functional-programming-oriented constructs). The trouble in python at the moment is that using something like `Maybe` and `Result` is not as nice to use as exceptions due to lack of a nice syntax i.e. for exceptions we have `try`/`except` and for functional error handling we’d need pattern matching.

Do `Maybe` and `Result` break duck typing? The 'returns' docs link to: https://sobolevn.me/2019/02/python-exceptions-considered-an-antipattern Something other than 'returns' would be easier to search/grep for.

On Sat, Sep 26, 2020 at 09:59:27AM +0200, Sascha Schlemmer via Python-ideas wrote:

I think a clear distinction has to be made between errors that belong to the api surface of a function e.g. some `ValueError` or `ZeroDivisionError` and *exceptional* errors e,g.`KeyboardInterrupt` or `DatabaseExplodedError`.

One problem is that while we can make a distinction between "expected" errors and "unexpected" errors, it's subjective and anything but clear. Whether you handle it, or just let it go, depends in part on the software you are writing.

For the first case I think a better way to model a function, that has some *expected* failure mode is to try and write a *total* function that doesn’t use exceptions at all. Let me show what I mean:

``` def div(a: int, b: int) -> float: return a / b ```

This function tries to divide two integers, trouble is that `div` is not defined for every input we may pass it, i.e. passing `b=0` will lead to `ZeroDivisionError` being raised. In other words `div` is a *partial* function because it is not defined for every member of its domain (type of its arguments).

There are two ways to amend this issue and make `div` a *total* function that is honest about its domain and co-domain (types of input arguments and return type):

You're making a mighty effort to bring some functional programming concepts to Python, but it doesn't really work. For example:

1. Extending the type of the functions co-domain (return type) and making the error case explicit [...]

``` def div(a: int, b: int) -> Maybe[float]: try: return Some(a / b) except ZeroDivisionError: return Nothing() ```

Alas, this still doesn't work because type annotations are advisory and not mandatory. Regardless of the type hints given, I can still call `div(1, "surprise")` and get a TypeError. Worse, because types in Python use subtyping, I can pass an int that does something you don't expect: class MyWeirdInt(int): def __truediv__(self, other): if other == 0: import loging logging.logg("tried to divide by " + other) raise ValueError('divided by zero') ... I've intentionally included at least three bugs in that method. Obviously in practice only one is likely to occur at a time, but this illustrates that even if the arguments to your div function are ints, in this case a subclass of int, you might expect a ZeroDivisionError but actually get: * ImportError (`import loging`) * AttributeError (`logging.logg`) * TypeError (concatenate a string with a number) * ValueError (explicit raise) or any other exception at all. So your "total function" is still not total: as soon as we move beyond the builtins written in C, literally anything can happen and any promise you make in `div` can be only advisory, not guaranteed. To get your "total function" you have to catch *anything*, but that cure is worse than the disease. That means that obvious bugs get suppressed and turned into `Nothing()` when they should in fact raise. And this brings us to where Java discovered that checked exceptions are counter-productive. To satisfy the compiler and have a "total function", anything which uses CheckedExceptions eventually ends up having the Java equivalent of: # Declared to only raise SomeError try: actual implementation except: # Catch anything and everything. raise SomeError("something bad happened") because that's the only way to end up with a total function in your sense, given that subclasses can raise anything. And that's why many of us are quite resistant to attempts to bring checked exceptions to Python. -- Steve

Alas, this still doesn't work because type annotations are advisory and not mandatory. Regardless of the type hints given, I can still call div(1, "surprise") and get a TypeError.

I'd assume that everyone is a consenting adult and if someone really wanted to shoot themselves in the foot there is no way to really prevent that (although mypy does a good job at that). The runtime `TypeError` at the call site of the offending code is actually what I'd prefer over code that mostly works (e.g. `div(1, 2)`) but crashes in the rare event that someone passes `b=0` to `div.`

Worse, because types in Python use subtyping, I can pass an int that does something you don't expect: class MyWeirdInt(int): def __truediv__(self, other): if other == 0: import loging logging.logg("tried to divide by " + other) raise ValueError('divided by zero')

Again there is no general way to prevent people from writing buggy code, the most can do is try and guide people towards using a library correctly (by good naming conventions, suitable levels of abstractions and yes also by making use of type hints)

So your "total function" is still not total: as soon as we move beyond the builtins written in C, literally anything can happen and any promise you make in div can be only advisory, not guaranteed. To get your "total function" you have to catch anything, but that cure is worse than the disease. That means that obvious bugs get suppressed and turned into Nothing() when they should in fact raise.

I fully agree with you that catching any possible exception just to make the compiler happy is the worst of all possible ways to handle errors. This is however not what I suggest; the point is not to make a function *total* in terms of anything that can happen in the interpreter/compile/runtime/hardware - that is what exceptions are for - but in terms of the functions api e.g. dividing by zero being a natural part of division that has to be handled somehow.