[Compiler-sig] Seperating out the parser

[Jeremy Hylton]

>>>>> "IB" == Ian Bicking <bickiia@earlham.edu> writes:

  IB> I've been reading through the source in Python-1.5.2/Parser, but
  IB> I was hoping to get a little help about where the border between
  IB> the compiler and the parser is.  Just a few pointers as to the
  IB> functions and datastructures that lie around that border, to
  IB> give me a place to start from.
  >>  The border between the parser and compiler is one of the key
  >> issues we need to sort out.  The Python compiler
  >> (Python/compile.c) uses the concrete parse tree generated by the
  >> parser.  The parser is generated from Grammar/Grammer using pgen.
  >> The compiler is a bit hard to read because it's referencing the
  >> parse tree nodes used integer indexes.

  IB> I'm afraid my knowledge of parsing/compiling is a little narrow,
  IB> so I'm not familiar with the distinction between concrete and
  IB> abstract parse trees.  But then, I've always tried to fudge my
  IB> way through parsing, and this is just another case of that since
  IB> I'm really just hoping to figure out how to *use* the already
  IB> exist parser(s) rather than implement one.  I guess I should
  IB> learn a little more about these things.  [and, having just
  IB> looked up the definition, I can see why an abstract syntax would
  IB> be much more useful]

  IB> The output from the parser module in Python creates something
  IB> more of a concrete parse tree, doesn't it?  It's called an AST,
  IB> but it seems to include things like NEWLINE, which seems rather
  IB> concrete.

You've got it exactly.  The concrete syntax includes things like
newlines and parens.  The abstract syntax does not.

Jeremy