[New-bugs-announce] [issue25555] Fix parser and AST: fill lineno and col_offset when compiling AST from Python objects
report at bugs.python.org
Thu Nov 5 04:42:04 EST 2015
New submission from STINNER Victor:
I wrote an AST optimizer which build AST from source code using ast.parse(), modify AST in place, and then compile the AST to bytecode.
The problem is that the lineno and col_offset attributes of the "arg" AST node are uninitialized when AST is rebuild from Python objects: compile(ast_tree, ...). As a consequence, the compilation may fail because lineno or col_offset values are invalid.
Attached patch enhances Parser/asdl_c.py to take "arg" attributes in account. It also updates the generated files Include/Python-ast.h and Python/Python-ast.c, and fix usage of the arg() function in Python/ast.c.
It looks like only the "arg" AST node had attributes which were initialized (other AST nodes are handled with a different code path which already filled attributes).
See Parser/Python.asdl for the definition of the "arg" AST node.
Note: Python 2.7 is not affected. In Python 2, function arguments are simple expressions, they don't have a dedicated type with lineno and col_offset attributes. "arg" was introduced in Python 3 with the PEP 3107 (function annotations).
nosy: benjamin.peterson, haypo, matrixise, yselivanov
title: Fix parser and AST: fill lineno and col_offset when compiling AST from Python objects
versions: Python 3.5, Python 3.6
Added file: http://bugs.python.org/file40949/ast.patch
Python tracker <report at bugs.python.org>
More information about the New-bugs-announce