Non-monotonically increasing line numbers in dis.findlinestarts() output

Consider this little session from the tip of the spear:
3 >> 8 LOAD_FAST 0 (a) 10 LOAD_CONST 2 (1) 12 INPLACE_SUBTRACT 14 STORE_FAST 0 (a) 2 16 LOAD_FAST 0 (a) 18 LOAD_CONST 1 (0) 20 COMPARE_OP 5 (>=) 22 POP_JUMP_IF_TRUE 8 4 >> 24 LOAD_FAST 0 (a) 26 RETURN_VALUE
list(dis.findlinestarts(while2.__code__)) [(0, 2), (8, 3), (16, 2), (24, 4)]
I get that somewhere along the way the compiler has been optimized to duplicate the loop's test so as to eliminate some jumps (perhaps). It's not clear to me that the line number should be "2" in the duplicated test though. Shouldn't it be "3"? I stumbled on this while trying to generate a line number table in my side project register VM. As I understand it, the line number delta in the output table is supposed to always be >= 0. In my code I'm using dis.findlinestarts() to determine the line numbers for each block. Perhaps I should be modifying its results. OTOH, maybe it's a bug. (If that's the consensus, I will create an issue.) Skip

Hi Skip, I modified co_lnotab format when I worked on the FAT Python optimization project which modified the AST to emit more efficient bytecode. My optimizer moved instructions, the line number could become non-monotonic (..., line 2, line 3, line 2, line 4, ...), and so lnotab (..., +1, -1, +2, ...) could be negative. Example with loop unrolling: def func1(): for i in ("a", "b"): print(i) is optimized: def func2(): i = "a" # line 1 print(i) # line 2 (+1) i = "b" # line 1 (-1) print(i) # line 2 (+1) I'm happy to see that Python 3.10 now also implements faster bytecode which rely on this change ;-) It seems like loop unrolling is not implemented in Python 3.10 yet. I guess that Python/ast_opt.c has no strategy yet to decide if an optimization should be applied or not depending on the code size. Loop unrolling can make code way bigger. Victor On Thu, Mar 18, 2021 at 12:12 AM Ned Batchelder <ned@nedbatchelder.com> wrote:
-- Night gathers, and now my watch begins. It shall not end until my death.

18.03.21 03:39, Victor Stinner пише:
I'm happy to see that Python 3.10 now also implements faster bytecode which rely on this change ;-)
It was used long time ago before 3.10. For example: a[i] = \ f() 2 0 LOAD_NAME 0 (f) 2 CALL_FUNCTION 0 1 4 LOAD_NAME 1 (a) 6 LOAD_NAME 2 (i) 8 STORE_SUBSCR 10 LOAD_CONST 0 (None) 12 RETURN_VALUE x = [ 1, 2, ] 2 0 LOAD_CONST 0 (1) 3 2 LOAD_CONST 1 (2) 1 4 BUILD_LIST 2 6 STORE_NAME 0 (x) 8 LOAD_CONST 2 (None) 10 RETURN_VALUE

Hi Skip, I modified co_lnotab format when I worked on the FAT Python optimization project which modified the AST to emit more efficient bytecode. My optimizer moved instructions, the line number could become non-monotonic (..., line 2, line 3, line 2, line 4, ...), and so lnotab (..., +1, -1, +2, ...) could be negative. Example with loop unrolling: def func1(): for i in ("a", "b"): print(i) is optimized: def func2(): i = "a" # line 1 print(i) # line 2 (+1) i = "b" # line 1 (-1) print(i) # line 2 (+1) I'm happy to see that Python 3.10 now also implements faster bytecode which rely on this change ;-) It seems like loop unrolling is not implemented in Python 3.10 yet. I guess that Python/ast_opt.c has no strategy yet to decide if an optimization should be applied or not depending on the code size. Loop unrolling can make code way bigger. Victor On Thu, Mar 18, 2021 at 12:12 AM Ned Batchelder <ned@nedbatchelder.com> wrote:
-- Night gathers, and now my watch begins. It shall not end until my death.

18.03.21 03:39, Victor Stinner пише:
I'm happy to see that Python 3.10 now also implements faster bytecode which rely on this change ;-)
It was used long time ago before 3.10. For example: a[i] = \ f() 2 0 LOAD_NAME 0 (f) 2 CALL_FUNCTION 0 1 4 LOAD_NAME 1 (a) 6 LOAD_NAME 2 (i) 8 STORE_SUBSCR 10 LOAD_CONST 0 (None) 12 RETURN_VALUE x = [ 1, 2, ] 2 0 LOAD_CONST 0 (1) 3 2 LOAD_CONST 1 (2) 1 4 BUILD_LIST 2 6 STORE_NAME 0 (x) 8 LOAD_CONST 2 (None) 10 RETURN_VALUE
participants (5)
-
MRAB
-
Ned Batchelder
-
Serhiy Storchaka
-
Skip Montanaro
-
Victor Stinner