better line number information in PyPy3
Hi all, This is a discussion mail about improving the traceback line number quality in PyPy3. Some context: CPython has started supporting negative line number offset from one bytecode to the next in its lnotab encoding, see here: https://bugs.python.org/issue26107 https://github.com/python/cpython/commit/f3914eb16d2#diff-cb296cc5109f5640ff... (Please don't ask me why not a single test was added for this new feature.) However, the support is so far only theoretically possible, the Python 3.7/3.8 bytecode compiler does not seem to make use of the feature. Negative line number jumps occur quite regularly with continuation lines, eg like this: def foobar(a, b): return "" + (1, a, b, 2 ) If you call this function, the traceback will contain this line: File "x.py", line 5, in foobar 2 TypeError: can only concatenate str (not "tuple") to str Which is obviously misleading. A slightly more realistic example are function decorators. There, the execution order is regularly "backwards": first the function definition is executed, then the individual decorators are run. There is currently a hack to sometimes leave the line number on the line of the first decorator, leading to confusing tracebacks. Example: def works(f): return f def broken(f): assert 0 @works @broken @works def g(): pass Running this, you'll get the following traceback: Traceback (most recent call last): File "y.py", line 9, in <module> @works File "y.py", line 5, in broken assert 0 AssertionError Pointing you again to the wrong line. We could fix this problem rather easily by removing the code that prevents the bytecode compiler from emitting negative line number offsets. Should I implement this? Advantages: - A lot less confusing tracebacks in real-world cases Disadvantages: - Different behavior from CPython (but arguably more correct) - Tracing tools could potentially be confused - Potentially larger lnotab strings due to more jumping around Cheers, Carl Friedrich
Hi Carl, I happened to stumble into this and saw your mail. This makes sense to me, rather cheap correctness is a good thing. Can we not at the same time negotiate with CPython to make their lineno info equally correct, maybe in 3.8 9 or at least 3.9? Cheers -- Chris On 05.06.19 10:57, Carl Friedrich Bolz wrote:
Hi all,
This is a discussion mail about improving the traceback line number quality in PyPy3. Some context:
CPython has started supporting negative line number offset from one bytecode to the next in its lnotab encoding, see here:
https://bugs.python.org/issue26107 https://github.com/python/cpython/commit/f3914eb16d2#diff-cb296cc5109f5640ff...
(Please don't ask me why not a single test was added for this new feature.)
However, the support is so far only theoretically possible, the Python 3.7/3.8 bytecode compiler does not seem to make use of the feature. Negative line number jumps occur quite regularly with continuation lines, eg like this:
def foobar(a, b): return "" + (1, a, b, 2 )
If you call this function, the traceback will contain this line:
File "x.py", line 5, in foobar 2 TypeError: can only concatenate str (not "tuple") to str
Which is obviously misleading.
A slightly more realistic example are function decorators. There, the execution order is regularly "backwards": first the function definition is executed, then the individual decorators are run. There is currently a hack to sometimes leave the line number on the line of the first decorator, leading to confusing tracebacks. Example:
def works(f): return f
def broken(f): assert 0
@works @broken @works def g(): pass
Running this, you'll get the following traceback:
Traceback (most recent call last): File "y.py", line 9, in <module> @works File "y.py", line 5, in broken assert 0 AssertionError
Pointing you again to the wrong line.
We could fix this problem rather easily by removing the code that prevents the bytecode compiler from emitting negative line number offsets. Should I implement this?
Advantages: - A lot less confusing tracebacks in real-world cases
Disadvantages: - Different behavior from CPython (but arguably more correct) - Tracing tools could potentially be confused - Potentially larger lnotab strings due to more jumping around
Cheers,
Carl Friedrich _______________________________________________ pypy-dev mailing list pypy-dev@python.org https://mail.python.org/mailman/listinfo/pypy-dev
-- Christian Tismer :^) tismer@stackless.com Software Consulting : http://www.stackless.com/ Karl-Liebknecht-Str. 121 : https://github.com/PySide 14482 Potsdam : GPG key -> 0xFB7BEE0E phone +49 173 24 18 776 fax +49 (30) 700143-0023
participants (2)
-
Carl Friedrich Bolz -
Christian Tismer