[Python-3000] Refactoring tool available (work in progress)

[Talin]

Guido van Rossum wrote:
> In the sandbox I've been working on a refactoring tool, which could
> form the basis for a Python 2.x -> 3.0 conversion tool.  I'd like to
> invite folks here to give it a try and give me a hand. It certainly
> needs more work, but I think that the basic infrastructure is sound.
> Check out sandbox/2to3/:
> http://svn.python.org/view/sandbox/trunk/2to3/.
> 
> This message is to invite feedback, and to encourage contributions. It
> would be great if people tried their hands at writing new
> transformations!

And here I thought Py3K wasn't going to have programmable syntax :)

I was working on a related idea recently, involving syntactical 
transformations, although not directly related to Python. One of the 
things I discovered was that it was possible in many cases to use the 
same parser on both the transformation patterns and the text to be 
translated. This allowed the transformation patterns to look a lot like 
the source and target patterns, instead of having to have a special 
syntax for the transformations.

Ideally, you ought to be able to say something like:

    apply(?f, ?x, ?y) => f(*x, **y)

You'd have to add an exception to the parser to recognize the unbound 
variable prefix ('?') when parsing patterns.

You could also put constraints on the bound variables:

    apply(?(expr:f), ?x, ?y) => (f)(*x, **y)
    apply(?f, ?x, ?y) => f(*x, **y)

In other words, if we need parens (because 'f' is an expression), then 
match the first rule, otherwise the second. Although a better way to 
handle the case of the parens is by repeated transformation:

    apply(?f, ?x, ?y) => paren(f)(*x, **y)
    paren(?(var:f)) => f
    paren(?f) => (f)

In this case, we transform 'f' into 'paren(f)', and then transform that 
into either 'f' or '(f)' depending on whether it's a simple variable or 
an expression. This allows us to only have one version of the 'apply' rule.

(Although the specific case of parenthesis logic ought to be built in, 
as you have already done, since it's going to clutter up every rule 
otherwise. Mainly I wanted to use it as an example to show constraints.)

What I'm describing is exactly the same logic as an algebraic solver, 
which many people have written in Python already.

One other thing you might want to do is look at the parser generator 
ANTLR (www.antlr.org), which has a syntax for tree transformations and 
pattern matching of tree fragments.

Anyway - it looks pretty cool :)

-- Talin