On Fri, 25 Sep 2020 at 18:28, Samuel Colvin <S@muelcolvin.com> wrote:

...

There's also the problem that you've explicitly acknowledged, that exception hints are *always* going to be inaccurate, in stark contrast to type hints which are expected to be correct when used.

We should draw a distinction between two kinds of "inaccurate": * inaccuracies because data about what errors could be raised is missing some exceptions * errors that could happen at almost any time: MemoryError, OverflowError, SystemError, SystemExit

The first set of errors is not in stark contrast to type hints, in fact they're very similar to errors in type hints which come up from time to time.

I get what you are saying, but I think you're being optimistic in assuming there's a clean distinction. It may be that you;'re thinking mostly about application code, where you are in control of what types exist in the system. But in library code, you simply cannot assume that. Consider a library that manipulates strings. There have been many attempts to write libraries that represent filesystem paths using a subclass of the string type. Suppose your library was passed one of those, and concatenated some text onto it. Possible exceptions that could in theory be raised include: * UnicodeError (if the path class checks that the value can be encoded in the filesystem encoding) * OSError (if the class enforces maximum path length or allowed character restrictions) * ValueError (if the programmer was lazy and used these rather than one of the above) as well as pretty much anything else, if the code that implemented __add__ for the path class had a bug. And of course, if libraries don't declare "expected" exceptions, application code using those libraries don't have anything to work with. Maybe you could get away with just declaring use of exception types that "you" created. But that's going to be very messy to get right.

Granted the "exceptions raised" database would be much more immature than the type hints database to start with. But that's not a reason not to start the error.

Agreed. What *is* a reason to not start is if we can't even agree what we're building.

The second set of exceptions are a different matter, I think in general they should not be included in the "exceptions raised" database. If that makes exceptions raised data invalid, then virtually all python logic in the world that catches some exceptions but doesn't catch those exceptions is also invalid!

Precisely. So it's critical to clearly define exactly what exceptions can happen "at any time", and given that operations like indexing, basic operators, and even attribute access can call arbitrary Python code, essentially any exception can happen at any time. Unless you start mandating exception behaviour on basic operations, and then you preclude creative but unusual designs such as the path class I described above. Maybe that's not unreasonable (over-clever path classes never really became that popular) but Python has a tradition of *not* arbitrarily constraining how features can be used.

In other words: right now when we write code and catch exceptions we generally (but not always) catch exceptions that are likely to happen in the general course of calling the code in the try, except block. But not all the strange things that could conceivably happen.

I think we should maintain that convention but provide a way to remind developers when they forget to catch an exception.

How do you distinguish between a developer "forgetting" to catch an exception and deliberately letting it bubble up. I'm a strong -1 on any proposal that makes people explicitly say "I meant to let this through" (it's essentially leading to checked exceptions, as well as violating the principle I stated above about not constraining how people use features - in this case the feature of exceptions bubbling up). And it's obviously impossible to make people declare when they forgot to catch an exception.

I guess overall, the question is: would it be useful to have a way to check what exceptions a block of code might raise?

Yes, massively.

You state that without any justification, and it's the key point. My view is that it's (barely) possible that it might be occasionally convenient, but no-one has really demonstrated a benefit that I care about at this point. And to be honest, claims like "tools might be able to warn about certain types of error" are too vague to carry much weight. I've been developing Python for many years, and work on pip, which is a major codebase that's full of legacy weirdness - and I don't think I've *ever* hit an issue where I've felt that having a checker warn me that I haven't caught a type of exception would be useful. (Many, many times I've had to fix issues where uncaught exceptions were raised, but they have basically always been logic errors where fixing the underlying issue is the correct action, not catching the exception. And that raises another point - often, uncaught exceptions are *better* than carefully catching everything. Better for the developer, not for the end user, but I'll come to that. If an uncaught exception happens, there's a full traceback that is often invaluable in debugging what went wrong. Certainly, a traceback is the worst sort of thing for the user to see, but I've debugged way too many systems where some bright spark has added an exception handler that converts exceptions to "user friendly" messages, and in doing so completely discards all of the context that I need to fix the issue. One potential risk with this proposal is that it will encourage a culture of thoughtlessly catching exceptions and "handling" them, often in a way that makes maintenance and support orders of magnitude harder. Consider the following: def some_function() -> None[raises AppError]: try: do() some() complicated() piece() of() work() except Exception as e: log.debug("Caught exception " + str(e)) raise AppError("Unexpected problem with some_function()") That looks like it's doing a good job of exception handling, but it would be a nightmare to debug. It hides the underlying error in a debug message (that the user might well not have enabled). It doesn't retain the call stack. It puts a huge chunk of code in a try...except block. Clearly it can be improved, and ideally someone with experience would point out the issues in code review. But in the real world, a junior developer, armed with a style guide that says "functions must declare the exceptions they could raise, and must log any unexpected problems" could have this in production before you could say "real life's never as ideal as you'd hope" :-)

Would it be a lot of work? yes, massive.

Would the work be worth the reward? Hard to say right now.

Unless someone defines the reward more explicitly, it's easy - no. Give me £100 and I might give you something in return. That's the proposition here at the moment... Sorry - this comes across as rather negative. Your arguments are well-stated and I appreciate the time you're taking to address the points I'm making. It's just that I remain unconvinced, I'm afraid. Paul