[Python-Dev] Range information in the AST -- once more

[Sven Brauch]

Hello!

I'm writing a static language analyzer for an IDE which reuses the
CPython parser (for parsing) [1]. Two years ago, I asked about a few
changes to be made to the AST provided by CPython, but the discussion
thread dried up before a definite decision was made. I decided to just
copy the parser code to my project and make the necessary changes
there back then. I'm bringing this up again now since after my project
has seen its first release recently, packagers are (understandably) a
bit unhappy about the python fork residing in its repository. I would
really like to get rid of that fork and link against the vanilla
libpython instead.

There's two things which are at the very least required to make this work:

1. The col_offset and lineno of an Attribute must give the beginning
of the word that names the attribute, not the beginning of the
expression. Example: In "foo.bar.baz", the col_offset of the Attribute
belonging to "bar" says "0" currently, it would need to be "4".

2. Column offsets and line numbers would need to be available for
function arguments (those with and without stars), and for alias
nodes.

In total, this requires very little change to the existing code, "a
few tens of lines changed at most" order of magnitude; those are
mostly trivial changes.
For what I can tell, the impact on existing code using the AST stuff
will be about zero. Even if there was some really obscure case where
the change would matter, porting would only require about three lines
of python code.

Additionally, there's a few more things which would be useful to have
available from the AST (namely the ranges of class and function names
when they are defined -- currently only the start of the first
decorator is available), but since those are reasonably easy to work
around it's not that important. It would still be nice tough.

If you think this is a reasonable suggestion then I'll be happy to
provide a patch for more detailed discussion.

Greetings,
Sven


________
[1] See https://projects.kde.org/kdev-python