Parsing and Editing Source

[Wilson]

On Aug 15, 3:45 pm, eliben <eli... at gmail.com> wrote:
> On Aug 15, 4:21 pm, "Paul Wilson" <paulalexwil... at gmail.com> wrote:
>
>
>
> > Hi all,
>
> > I'd like to be able to do the following to a python source file
> > programmatically:
> >  * Read in a source file
> >  * Add/Remove/Edit Classes, methods, functions
> >  * Add/Remove/Edit Decorators
> >  * List the Classes
> >  * List the imported modules
> >  * List the functions
> >  * List methods of classes
>
> > And then save out the result back to the original file (or elsewhere).
>
> > I've begun by using the tokenize module to generate a token-tuple list
> > and am building datastructures around it that enable the above
> > methods. I'm find that I'm getting a little caught up in the details
> > and thought I'd step back and ask if there's a more elegant way to
> > approach this, or if anyone knows a library that could assist.
>
> > So far, I've got code that generates a line number to token-tuple list
> > dictionary, and am working on a datastructure describing where the
> > classes begin and end, indexed by their name, such that they can be
> > later modified.
>
> > Any thoughts?
> > Thanks,
> > Paul
>
> Consider using the 'compiler' module which will lend you more help
> than 'tokenize'.
>
> For example, the following demo lists all the method names in a file:
>
> import compiler
>
> class MethodFinder:
>     """ Print the names of all the methods
>
>         Each visit method takes two arguments, the node and its
>         current scope.
>         The scope is the name of the current class or None.
>     """
>
>     def visitClass(self, node, scope=None):
>         self.visit(node.code, node.name)
>
>     def visitFunction(self, node, scope=None):
>         if scope is not None:
>             print "%s.%s" % (scope, node.name)
>         self.visit(node.code, None)
>
> def main(files):
>     mf = MethodFinder()
>     for file in files:
>         f = open(file)
>         buf = f.read()
>         f.close()
>         ast = compiler.parse(buf)
>         compiler.walk(ast, mf)
>
> if __name__ == "__main__":
>     import pprint
>     import sys
>
>     main(sys.argv)

Thanks! Will I be able to make changes to the ast such as "rename
decorator", "add decorator", etc.. and write them back out to a file
as Python source?

Regards,
Paul