[pypy-dev] Re: Project suggestions

[Armin Rigo]

Hi Aurelien,

On Sun, Oct 02, 2005 at 03:02:48AM +0200, Aur?lien Camp?as wrote:
> >Instead, we rewrite support for the infinitely simpler RPython
> >semantics, and even then we try to share code between back-ends (which
> >is the purpose of the RTyper).  Using the RTyper is not mandatory for
> >back-ends, but you definitely cannot analyse RPython code from the AST
> >for the good reason that it's a language without source code.
> 
> Can you expand on this ? I tend to see the AST as a programmatically 
> manageable representation of the source code. Is this expectation wrong ?

No, you are right, but RPython has neither a source nor an AST
representation:

* first, we take a normal Python program and we import it in CPython.
  During the import, lots of initialization stuff happens.  We may even
  call more initialization functions that will set up more stuff.  This
  phase creates module, class, function and even user instances -- e.g.
  in the case where the normal Python program is the PyPy interpreter,
  we build a complete object space during this phase, which is highly
  non-trivial and creates a lot of instances of our own classes,
  referencing each other in complex ways.

* if all this is carefully done, then the resulting set of objects
  (classes, functions, prebuilt instances) may conform to the RPython
  rules.  This is what an "RPython program" really is: it's a point in
  time, a snapshot, a set of objects referencing each other in such a
  way that the functions left in this set of objects will no longer
  perform dynamic stuff after this point in time.

An RPython program has no immediate syntactic equivalent.  You can
consider that each function individually has got an AST, but there is no
*global* or even module-level AST to translate.  In PyPy, instead of the
AST, we turn each function into a flow graph and look at that.  We
perform the "annotation" (type inference) on the family of all flow
graphs.

> I need an understandable entry point anyway, so beginning with the
> "easy" stuff and rediscovering already known mistakes is somewhat
> unavoidable ; it'll help to understand the rationale behind Pypy's
> architecture.

Indeed.  I guess I have gone too quickly to too obscure considerations.
In one sentence, I should have said that it makes sense in PyPy to start
from annotated flow graphs like gencl and genpyrex do, instead of
considering ASTs, because it's the approach that can eventually evolve
into a complete RPython solution with the help of other parts of PyPy.
At this point it definitely makes sense to rebuild a CL backend from
scratch to get the feeling of flow graphs.


A bientot,

Armin.