Non-monotonically increasing line numbers in dis.findlinestarts() output
Consider this little session from the tip of the spear:
sys.version '3.10.0a6+ (heads/master:0ab152c6b5, Mar 15 2021, 17:24:38) [GCC 10.2.0]' def while2(a): ... while a >= 0: ... a -= 1 ... return a ... dis.dis(while2) 2 0 LOAD_FAST 0 (a) 2 LOAD_CONST 1 (0) 4 COMPARE_OP 5 (>=) 6 POP_JUMP_IF_FALSE 24
3 >> 8 LOAD_FAST 0 (a) 10 LOAD_CONST 2 (1) 12 INPLACE_SUBTRACT 14 STORE_FAST 0 (a) 2 16 LOAD_FAST 0 (a) 18 LOAD_CONST 1 (0) 20 COMPARE_OP 5 (>=) 22 POP_JUMP_IF_TRUE 8 4 >> 24 LOAD_FAST 0 (a) 26 RETURN_VALUE
list(dis.findlinestarts(while2.__code__)) [(0, 2), (8, 3), (16, 2), (24, 4)]
I get that somewhere along the way the compiler has been optimized to duplicate the loop's test so as to eliminate some jumps (perhaps). It's not clear to me that the line number should be "2" in the duplicated test though. Shouldn't it be "3"? I stumbled on this while trying to generate a line number table in my side project register VM. As I understand it, the line number delta in the output table is supposed to always be >= 0. In my code I'm using dis.findlinestarts() to determine the line numbers for each block. Perhaps I should be modifying its results. OTOH, maybe it's a bug. (If that's the consensus, I will create an issue.) Skip
On 2021-03-17 22:10, Skip Montanaro wrote:
Consider this little session from the tip of the spear:
sys.version '3.10.0a6+ (heads/master:0ab152c6b5, Mar 15 2021, 17:24:38) [GCC 10.2.0]' def while2(a): ... while a >= 0: ... a -= 1 ... return a ... dis.dis(while2) 2 0 LOAD_FAST 0 (a) 2 LOAD_CONST 1 (0) 4 COMPARE_OP 5 (>=) 6 POP_JUMP_IF_FALSE 24
3 >> 8 LOAD_FAST 0 (a) 10 LOAD_CONST 2 (1) 12 INPLACE_SUBTRACT 14 STORE_FAST 0 (a)
2 16 LOAD_FAST 0 (a) 18 LOAD_CONST 1 (0) 20 COMPARE_OP 5 (>=) 22 POP_JUMP_IF_TRUE 8
4 >> 24 LOAD_FAST 0 (a) 26 RETURN_VALUE
list(dis.findlinestarts(while2.__code__)) [(0, 2), (8, 3), (16, 2), (24, 4)]
I get that somewhere along the way the compiler has been optimized to duplicate the loop's testso as to eliminate some jumps (perhaps). It's not clear to me that the line number should be "2" in the duplicated test though.Shouldn't it be "3"?
The condition is on line 2. The fact that it has been duplicated n the bytecode is irrelevant; it's still on line 2 in the source. Line 3 has the decrement and line 4 has the return.
I stumbled on this while trying to generate a line number table in my side project register VM. As I understand it, the line number delta in the output table is supposed to always be >= 0. In my code I'm using dis.findlinestarts() to determine the line numbers for each block. Perhaps I should be modifying its results. OTOH, maybe it's a bug. (If that's the consensus, I will create an issue.)
On 3/17/21 6:41 PM, MRAB wrote:
On 2021-03-17 22:10, Skip Montanaro wrote:
I stumbled on this while trying to generate a line number table in my side project register VM. As I understand it, the line number delta in the output table is supposed to always be >= 0. In my code I'm using dis.findlinestarts() to determine the line numbers for each block. Perhaps I should be modifying its results. OTOH, maybe it's a bug. (If that's the consensus, I will create an issue.)
co_lnotab has had negative deltas since 3.6. --Ned.
Hi Skip, I modified co_lnotab format when I worked on the FAT Python optimization project which modified the AST to emit more efficient bytecode. My optimizer moved instructions, the line number could become non-monotonic (..., line 2, line 3, line 2, line 4, ...), and so lnotab (..., +1, -1, +2, ...) could be negative. Example with loop unrolling: def func1(): for i in ("a", "b"): print(i) is optimized: def func2(): i = "a" # line 1 print(i) # line 2 (+1) i = "b" # line 1 (-1) print(i) # line 2 (+1) I'm happy to see that Python 3.10 now also implements faster bytecode which rely on this change ;-) It seems like loop unrolling is not implemented in Python 3.10 yet. I guess that Python/ast_opt.c has no strategy yet to decide if an optimization should be applied or not depending on the code size. Loop unrolling can make code way bigger. Victor On Thu, Mar 18, 2021 at 12:12 AM Ned Batchelder <ned@nedbatchelder.com> wrote:
On 3/17/21 6:41 PM, MRAB wrote:
On 2021-03-17 22:10, Skip Montanaro wrote:
I stumbled on this while trying to generate a line number table in my side project register VM. As I understand it, the line number delta in the output table is supposed to always be >= 0. In my code I'm using dis.findlinestarts() to determine the line numbers for each block. Perhaps I should be modifying its results. OTOH, maybe it's a bug. (If that's the consensus, I will create an issue.)
co_lnotab has had negative deltas since 3.6.
--Ned. _______________________________________________ Python-Dev mailing list -- python-dev@python.org To unsubscribe send an email to python-dev-leave@python.org https://mail.python.org/mailman3/lists/python-dev.python.org/ Message archived at https://mail.python.org/archives/list/python-dev@python.org/message/EK2O6SE6... Code of Conduct: http://python.org/psf/codeofconduct/
-- Night gathers, and now my watch begins. It shall not end until my death.
18.03.21 03:39, Victor Stinner пише:
I'm happy to see that Python 3.10 now also implements faster bytecode which rely on this change ;-)
It was used long time ago before 3.10. For example: a[i] = \ f() 2 0 LOAD_NAME 0 (f) 2 CALL_FUNCTION 0 1 4 LOAD_NAME 1 (a) 6 LOAD_NAME 2 (i) 8 STORE_SUBSCR 10 LOAD_CONST 0 (None) 12 RETURN_VALUE x = [ 1, 2, ] 2 0 LOAD_CONST 0 (1) 3 2 LOAD_CONST 1 (2) 1 4 BUILD_LIST 2 6 STORE_NAME 0 (x) 8 LOAD_CONST 2 (None) 10 RETURN_VALUE
participants (5)
-
MRAB -
Ned Batchelder -
Serhiy Storchaka -
Skip Montanaro -
Victor Stinner