[New-bugs-announce] [issue45757] dis module incorrectly handles EXTENDED_ARG + NOP sequence

[Rok Mandeljc]

New submission from Rok Mandeljc <rok.mandeljc at gmail.com>:

dis module incorrectly handles the instruction sequence where EXTENDED_ARG is followed by NOP, e.g.,:

EXTENDED_ARG 0x01
NOP
EXTENDED_ARG 0x01
LOAD_CONST 0x29

The above sequence loads the constant from index 0x0129, whereas dis attempts to load it from index 0x010129, resulting in "IndexError: tuple index out of range error" when attempting to iterate over instructions returned by dis.get_instructions().

Complete example:

# *** example.py begin ***
import types
import dis

# Create a test code object...
constants = [None] * (0x000129 + 1)
constants[0x000129] = "Hello world!"

code = types.CodeType(
    0,  # argcount
    0,  # posonlyargcount
    0,  # kwonlyargcount
    0,  # nlocals
    1,  # stacksize
    64,  # flags
    bytes([
        0x90, 0x01,  # EXTENDED_ARG 0x01
        0x09, 0xFF,  # NOP 0xFF
        0x90, 0x01,  # EXTENDED_ARG 0x01
        0x64, 0x29,  # LOAD_CONST 0x29
        0x53, 0x00,  # RETURN_VALUE 0x00
    ]),  # codestring=
    tuple(constants),  # constants
    (),  # names
    (),  # varnames
    '<no file>',  # filename
    'code',  # name
    1,  # firstlineno
    b''  # linetable
)

# ... and eval it to show that NOP resets EXTENDED_ARG
print("Output:", eval(code))

# Try to list all instructions via dis
print(list(dis.get_instructions(code)))

# *** example.py end ***

Running the example gives us:

Output: Hello world!
Traceback (most recent call last):
  File "/home/rok/example.py", line 35, in <module>
    print(list(dis.get_instructions(code)))
  File "/usr/lib64/python3.10/dis.py", line 338, in _get_instructions_bytes
    argval, argrepr = _get_const_info(arg, constants)
  File "/usr/lib64/python3.10/dis.py", line 292, in _get_const_info
    argval = const_list[const_index]
IndexError: tuple index out of range


To fix the problem on dis side, the extended_arg in dis._unpack_opargs should be reset to 0 when NOP (or perhaps any opcode without argument) is encountered. I.e., by adding "extended_arg = 0" here:
https://github.com/python/cpython/blob/91275207296c39e495fe118019a757c4ddefede8/Lib/dis.py#L525



The problematic behavior was reported by users of PyInstaller under python 3.10; it seems that python 3.10 generates bytecode involving EXTENDED_ARG + NOP for telegram.message.py:
https://raw.githubusercontent.com/python-telegram-bot/python-telegram-bot/v13.7/telegram/message.py

The original PyInstaller issue with preliminary analysis is here: https://github.com/pyinstaller/pyinstaller/issues/6301

----------
components: Library (Lib)
messages: 405985
nosy: rok.mandeljc
priority: normal
severity: normal
status: open
title: dis module incorrectly handles EXTENDED_ARG + NOP sequence
type: behavior
versions: Python 3.10, Python 3.8, Python 3.9

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue45757>
_______________________________________