[pypy-dev] better line number information in PyPy3

[Christian Tismer]

Hi Carl,

I happened to stumble into this and saw your mail.

This makes sense to me, rather cheap correctness is a good thing.
Can we not at the same time negotiate with CPython to make
their lineno info equally correct, maybe in 3.8 9 or at least 3.9?

Cheers -- Chris


On 05.06.19 10:57, Carl Friedrich Bolz wrote:
> Hi all,
> 
> This is a discussion mail about improving the traceback line number
> quality in PyPy3. Some context:
> 
> CPython has started supporting negative line number offset from one
> bytecode to the next in its lnotab encoding, see here:
> 
> https://bugs.python.org/issue26107
> https://github.com/python/cpython/commit/f3914eb16d2#diff-cb296cc5109f5640ff3f6d7198a6abee
> 
> (Please don't ask me why not a single test was added for this new feature.)
> 
> However, the support is so far only theoretically possible, the Python
> 3.7/3.8 bytecode compiler does not seem to make use of the feature.
> Negative line number jumps occur quite regularly with continuation
> lines, eg like this:
> 
> def foobar(a, b):
>     return "" + (1,
>             a,
>             b,
>             2
>             )
> 
> If you call this function, the traceback will contain this line:
> 
>   File "x.py", line 5, in foobar
>     2
> TypeError: can only concatenate str (not "tuple") to str
> 
> Which is obviously misleading.
> 
> 
> A slightly more realistic example are function decorators. There, the
> execution order is regularly "backwards": first the function
> definition is executed, then the individual decorators are run. There
> is currently a hack to sometimes leave the line number on the line of
> the first decorator, leading to confusing tracebacks. Example:
> 
> def works(f):
>     return f
> 
> def broken(f):
>     assert 0
> 
> @works
> @broken
> @works
> def g():
>     pass
> 
> 
> Running this, you'll get the following traceback:
> 
> Traceback (most recent call last):
>   File "y.py", line 9, in <module>
>     @works
>   File "y.py", line 5, in broken
>     assert 0
> AssertionError
> 
> Pointing you again to the wrong line.
> 
> 
> We could fix this problem rather easily by removing the code that
> prevents the bytecode compiler from emitting negative line number
> offsets. Should I implement this?
> 
> Advantages:
> - A lot less confusing tracebacks in real-world cases
> 
> Disadvantages:
> - Different behavior from CPython (but arguably more correct)
> - Tracing tools could potentially be confused
> - Potentially larger lnotab strings due to more jumping around
> 
> Cheers,
> 
> Carl Friedrich
> _______________________________________________
> pypy-dev mailing list
> pypy-dev at python.org
> https://mail.python.org/mailman/listinfo/pypy-dev
> 


-- 
Christian Tismer             :^)   tismer at stackless.com
Software Consulting          :     http://www.stackless.com/
Karl-Liebknecht-Str. 121     :     https://github.com/PySide
14482 Potsdam                :     GPG key -> 0xFB7BEE0E
phone +49 173 24 18 776  fax +49 (30) 700143-0023