[pypy-dev] Re: Notes on compiler package
mwh at python.net
Thu Jan 23 12:22:58 CET 2003
logistix <logistix at zworg.com> writes:
> I'm finally starting to get my head wrapped around the compiler package,
> and thought I'd pass some notes on to the group since it's a beast:
As the person who did some fixing up so it worked in 2.3a1, I second
> 1) Other than the original AST generation, it looks like it's all pure
> python (not tying into the C source). The original AST comes from the
> parser module, which is a pure wrapper module (although I've already
> posted python code to build ast's here)
> 2) Internal CPython ASTs are transformed into "compiler"'s own ASTs.
> These nodes are smarter than the internal ones; they attach appropriate
> info to "attributes". CPython AST nodes just have a list of children,
> this module identifies and breaks out important data (for example, the
> arguments in a function def become an "args" attribute instead of
> guessing it's the first child)
Yep. This is compiler.transformer. It has to be said, if you were
writing a parser for Python in Python, I'd shoot for generating the
output of the transformer module rather than aping what the parser
> 3) The CodeGenerators walk the ASTs and emit code into a code object.
> 4) Speed seems reasonable (which makes me think I'm missing some calls
> into the C side of things)
You obviously haven't tried to compile anything big with it yet. It's
> 4) The generated bytecode looks reasonably good and execs fine, but
> there are still some diffences from what CPython generates. Here's what
> I've seen so far:
> SET_LINENO doesn't always work the same (this is how python knows
> what line in a file threw an exception)
Hey, in 2.3 SET_LINENO doesn't even exist any more :-)
> Doesn't have one of the few optimizations the CPython compiler does
> have. CPython will throw out misplaced "docstrings"... strings in the
> source that aren't assigned to anything. This throws off array indexes
> in the code objects co_const attribute.
Yeah, indices into co_consts are different (and not just in the case
you mention above, I think). But it works -- you can compile the
stdlib with in and run the test suite happily.
> In general, the compiler package is in alot better shape than I expected.
What were you expecting?
More information about the Pypy-dev