[Python-3000] Refactoring tool available (work in progress)

[Guido van Rossum]

I'm taking the idea of using the same parser with a grain of salt --
by the time you have added all the pattern matching syntax IMO it will
look too cluttered. Also, your concept of "variable" is too
constrained -- the parenthesization should not only depend on the form
of the experssion, but also on the context. E.g. "if x.has_key(a+b):"
can translate to "if a+b in x:", but "3*x.has_key(a+b)" has to
translate to "3*(a+b in x)".

I'm not sure how thinking about it as an algebraic solver helps -- is
there any existing code I can borrow?

I looked at this bit of ANTLR briefly:
http://www.antlr.org/doc/sor.html#_bb8; from the example it looks like
their notation isn't any more compact than the same thing written in
Python (despite ANTLR's over-use of line-noise characters). In fact, I
can probably write that example using my tool easily: the pattern is
something like

  arith_expr<t=term '+' n=NUMBER | n=NUMBER '+' t=term>

and the transformation (after setting up t and n to point to the
respective subtrees) could be something like

  try:
    value = int(n)
  except ValueError:
    return None
  if value != 0:
    return None
  return t.clone()

--Guido

On 12/14/06, Talin <talin at acm.org> wrote:
> Guido van Rossum wrote:
> > In the sandbox I've been working on a refactoring tool, which could
> > form the basis for a Python 2.x -> 3.0 conversion tool.  I'd like to
> > invite folks here to give it a try and give me a hand. It certainly
> > needs more work, but I think that the basic infrastructure is sound.
> > Check out sandbox/2to3/:
> > http://svn.python.org/view/sandbox/trunk/2to3/.
> >
> > This message is to invite feedback, and to encourage contributions. It
> > would be great if people tried their hands at writing new
> > transformations!
>
> And here I thought Py3K wasn't going to have programmable syntax :)
>
> I was working on a related idea recently, involving syntactical
> transformations, although not directly related to Python. One of the
> things I discovered was that it was possible in many cases to use the
> same parser on both the transformation patterns and the text to be
> translated. This allowed the transformation patterns to look a lot like
> the source and target patterns, instead of having to have a special
> syntax for the transformations.
>
> Ideally, you ought to be able to say something like:
>
>     apply(?f, ?x, ?y) => f(*x, **y)
>
> You'd have to add an exception to the parser to recognize the unbound
> variable prefix ('?') when parsing patterns.
>
> You could also put constraints on the bound variables:
>
>     apply(?(expr:f), ?x, ?y) => (f)(*x, **y)
>     apply(?f, ?x, ?y) => f(*x, **y)
>
> In other words, if we need parens (because 'f' is an expression), then
> match the first rule, otherwise the second. Although a better way to
> handle the case of the parens is by repeated transformation:
>
>     apply(?f, ?x, ?y) => paren(f)(*x, **y)
>     paren(?(var:f)) => f
>     paren(?f) => (f)
>
> In this case, we transform 'f' into 'paren(f)', and then transform that
> into either 'f' or '(f)' depending on whether it's a simple variable or
> an expression. This allows us to only have one version of the 'apply' rule.
>
> (Although the specific case of parenthesis logic ought to be built in,
> as you have already done, since it's going to clutter up every rule
> otherwise. Mainly I wanted to use it as an example to show constraints.)
>
> What I'm describing is exactly the same logic as an algebraic solver,
> which many people have written in Python already.
>
> One other thing you might want to do is look at the parser generator
> ANTLR (www.antlr.org), which has a syntax for tree transformations and
> pattern matching of tree fragments.
>
> Anyway - it looks pretty cool :)
>
> -- Talin
> _______________________________________________
> Python-3000 mailing list
> Python-3000 at python.org
> http://mail.python.org/mailman/listinfo/python-3000
> Unsubscribe: http://mail.python.org/mailman/options/python-3000/guido%40python.org
>


-- 
--Guido van Rossum (home page: http://www.python.org/~guido/)