On Tue, Apr 20, 2021 at 2:48 AM Nathaniel Smith <njs@pobox.com> wrote:

The problem is that most of the time, even if you're using concurrency
internally so multiple things *could* go wrong at once, only one thing
actually *does* go wrong. So it's unfortunate if some existing code is
prepared for a specific exception that it knows can be raised, that
exact exception is raised... and the existing code fails to catch it
because now it's wrapped in an EG.

Yes, this was discussed at length on this list. Raising an exception group is an API-breaking change. If a function starts raising exception groups its callers need to be prepared for that. Realistically we think exception groups will be raised by new APIs.  We tried and were unable to define exception group semantics for except that would be reasonable and backwards compatible. That's why we added except*.


> It is easy enough to write a denormalize() function in traceback.py that constructs this from the current EG structure, if you need it (use the leaf_generator from the PEP). I'm not sure I see why we should trouble the interpreter with this.

In the current design, once an exception is wrapped in an EG, then it
can never be unwrapped, because its traceback information is spread
across the individual exception + the EG tree around it. This is
confusing to users ("how do I check errno?"), and makes the design
more complicated (the need for topology-preserving .split(), the
inability to define a sensible EG.__iter__, ...). The advantage of
making the denormalized form the native form is that now the leaf
exceptions would be self-contained objects like they are now, so you
don't need EG nesting at all, and users can write intuitive code like:

except OSError as *excs:
    remainder = [exc for exc in excs if exc.errno != ...]
    if remainder:
        raise ExceptionGroup(remainder)

We have this precise example in the PEP:
   match, rest = excs.split(lambda e: e.errno != ...)

You use split() instead of iteration for that.  split() preserves all __context__, __cause__ and __traceback__ information, on all leaf and non-leaf exceptions.
 

> For display purposes, it is probably nicer to look at a normalized traceback where common parts are not repeated.

Yeah, I agree; display code would want to re-normalize before
printing. But now it's only the display code that needs to care about
figuring out shared parts of the traceback, rather than something that
has to be maintained as an invariant everywhere.

If you *do* want iteration, we show in the PEP how to write a leaf_generator() that gives you the leaf exceptions with their tracebacks (as a list of chunks). It is easy to copy the chunks into a single flat traceback. We didn't propose to add it to traceback.py yet because the use case is unclear but if people need it we're talking about 10-15 lines in traceback.py.

So for your suggestion:

Pros:
1. Those who want a denormalized traceback (if they exist) won't need to call traceback.denormalize().

Cons:  
1. A significant change in the interpreter that will make it less efficient (both time and space).
2. Display code will need to normalize the traceback, which is much more complicated than denormalizing because you need to discover the shared parts.

Am I missing something?


>
> It sounds like you want some way to enrich exceptions. This should be optional (we wouldn't want EG to require additional metadata for exceptions) so yeah, I agree it should sit on the leaf exception and not on the group. In that sense it's orthogonal to this PEP.

Well, the extra metadata would specifically be at "join" points in the
traceback, which are a thing that this PEP is creating :-). And it's
useful for every user of EGs, since by definition, an EG is
multiplexing exceptions from multiple sources, so it's nice to tell
the user which sources those are. 
That said, you're right, if we want to handle this by defining a new
kind of traceback entry that code like Trio/asyncio/hypothesis can
manually attach to exceptions, then that could be written as a
separate-but-complementary PEP. In my original design, instead of
defining a new kind of traceback entry, I was storing this on the EG
itself, so that's why I was thinking about it needing to be part of
this PEP.

You can also create an ExceptionGroup subclass with whatever extra data you want to include.
 
 Irit