[Python-ideas] Shrink recursion error tracebacks (was: Have REPL print less by default)

[Franklin? Lee]

(Sorry, my previous message was sent from the wrong address, so only
Émanuel got it. I then replied to his reply, which had me reuse the
wrong address. Sorry, Émanuel, for this repeat.)

On Fri, Apr 22, 2016 at 10:00 PM, Émanuel Barry <vgr255 at live.ca> wrote:
> I have been thinking, and especially after Nick’s comments, that it might be
> better to keep it as simple as possible to reduce the risk of errors
> happening while we’re printing the tracebacks. Recursive functions (that go
> deep enough to trigger an exception) are usually within one function, in the
> REPL, or by using __getattr__ / __getattribute__. Nice to have? Sure.
> Necessary? Don’t think as much.

Simple as in, unlikely to be affected by whatever caused the exception
in the first place?

Here's pseudocode for my suggestion.
(I assume appropriate definitions of `traceback`, `print_folded`, and
`print_the_thing`. I assume `func_name` is a qualified name (e.g.
`('f', '<stdin>')`).)

    seen = collections.Counter() # Counts number of times each
(func,line_no) has been seen
    block = None # A block of potentially-hidden functions.
    prev_line_no = None # In case len(block) == 1.
    hidecount = 0 # Total number of hidden lines.
    for func_name, line_no in traceback:
        times = seen[func_name, line_no] += 1
        if times >= 3:
            if block is None:
                block = collections.Counter()
            block[func_name] += 1
            prev_line_no = line_no
        else:
            # This `if` can be a function which returns a hidecount,
            #  so we don't repeat ourselves at the end of the loop.
            if block is not None:
                if len(block) == 1: # don't need to hide
                    print_the_thing(next(block.
keys()), prev_line_no)
                else:
                    print_folded(block)
                    hidecount += len(block)
                block = None
            print_the_thing(func_name, line_no)

    if block is not None:
        if len(block) == 1:
            print_the_thing(block[0])
        else:
            print_folded(block)
            hidecount += len(block)


> I’m also a very solid -1 on the idea of prompting the user to do something
> like full_ex() to get the full stack trace. My rationale for such a change
> is that we need to be extremely precise in our message, to leave absolutely
> no room for a different interpretation than what we mean. Your message, for
> instance, is ambiguous. The fact it says that calls are hidden would let me
> think I just lost information about the stack trace (though that’s but a
> wording issue). As a user, just seeing "Call _full_ex() to print full
> trace." would be an immediate red flag: Crap, I just lost precious debugging
> information to save on lines!

My hiding is more complex (can't reproduce original output exactly),
so it would be important to have an obvious way to get the old
behavior. Someone else can propose the wording, if the hiding strategy
itself seems useful.


On Fri, Apr 22, 2016 at 11:50 PM, Steven D'Aprano <steve at pearwood.info> wrote:
>
> For the record, detecting subsequences of mutally recurive calls is not
> a trivial task. (It's not the same as cycle detection.) And I don't
> understand how you can get 300 calls to f and 360 calls to g in a
> scenario where f calls g and g calls f. Surely there would be equal
> number of calls to each?

It depends on how you define the problem. I look for, "contiguous
block of output, each line of which has been seen before."

(You can set up a trivial example where g calls itself 60 times if the
parameter is mod 300. Other examples might involve randomness, or
perhaps it's just a really deep recursion rather than a cycle. I'm
just giving an example of how you don't need to detect repeated
blocks.)