Extracting attributes from compiled python code or parse trees

Matteo mahall at ncsa.uiuc.edu
Mon Jul 23 23:13:05 CEST 2007


Hello-
I am trying to get Python to extract attributes in full dotted form
from compiled expression. For instance, if I have the following:

param = compile('a.x + a.y','','single')

then I would like to retrieve the list consisting of ['a.x','a.y'].
I have tried using inspect to look at 'co_names', but when I do that,
I get:

>>> inspect.getmembers(param)[23]
('co_names', ('a', 'x', 'y'))

with no way to determine that 'x' and 'y' are both attributes of 'a'.

The reason I am attempting this is to try and automatically determine
data dependencies in a user-supplied formula (in order to build a
dataflow network). I would prefer not to have to write my own parser
just yet.

Alternatively, I've looked at the parser module, but I am experiencing
some difficulties in that the symbol list does not seem to match that
listed in the python grammar reference (not surprising, since I am
using python2.5, and the docs seem a bit dated)

In particular:

>>> import parser
>>> import pprint
>>> import symbol
>>> tl=parser.expr("a.x").tolist()
>>> pprint.pprint(tl)

[258,
 [326,
  [303,
   [304,
    [305,
     [306,
      [307,
       [309,
        [310,
         [311,
          [312,
           [313,
            [314,
             [315,
              [316, [317, [1, 'a']], [321, [23, '.'], [1,
'x']]]]]]]]]]]]]]]],
 [4, ''],
 [0, '']]

>>> print symbol.sym_name[316]
power

Thus, for some reason, 'a.x' seems to be interpreted as a power
expression, and not an 'attributeref' as I would have anticipated (in
fact, the symbol module does not seem to contain an 'attributeref'
symbol)

(for the curious, here is the relevant part of the AST for "a**x":
            [316,
               [317, [1, 'a']],
               [36, '**'],
               [315, [316, [317, [1, 'x']]]]
)

Anyway, I could write an AST analyzer that searches for the correct
pattern, but it would be relying on undocumented behavior, and I'm
hoping there is a better way.

(By the way, I realize that malicious users could almost certainly
subvert my proposed dependency mechanism, but for this project, I'm
guarding against Murphy, not Macchiavelli)

Thanks,
-matt




More information about the Python-list mailing list