compiler package vs parser

[Robin Becker]

Kay Schluehr wrote:
> On 16 Apr., 11:41, Robin Becker <ro... at reportlab.com> wrote:
> 
>> Is the compiler package actually supposed to be equivalent to the parser module?
> 
> No. The parser module creates a concrete parse tree ( CST ) whereas
> the compiler package transforms this CST into an AST for subsequent
> computations. In more recent versions those CST -> AST transformations
> are performed by the runtime and the Python compiler uses those
> internally produced ASTs. The Python 2.6 API to ASTs is the ast module.
......

OK having digested most of what Kay said (my simple engineer's brain being 
rather stretched at the best of times) I would like to rephrase my question.

Is it intended that the compile package be able to create the same code as the 
normal parser chain. I think I understand now that the compiler used internally 
is now exposed in various ways as built in modules.

The example I gave earlier seems to indicate that the compiler package may not 
be creating the same final bytecode at least using the compilation approach that 
I used (cbased on the compiler package itself).

If I have messed that up then there should be some easy fix, otherwise if 
pycodegen is somehow not getting the semantics of the the variables i,j correct 
is there some way I can fix that.

I realize that I probably ought to be trying this out with the newer ast stuff, 
but currently I am supporting code back to 2.3 and there's not much hope of 
doing it right there without using the compiler package.

My analysis of the problem is that in

#### start p.py
def func(D):
     for k in D:
         exec '%s=D[%r]' % (k,k)
     print i, j, k
     print locals()
     print i, j, k

if __name__=='__main__':
     func(dict(i=1,j=33))
#### end p.py

the compiler package ends up treating i & j as global, whereas the modern 
analysis doesn't (and doesn't say they're definitely local either). Looking at 
the code in Python/compile.c the compile_nameop code seems to check for scopes 
FREE, CELL, LOCAL, GLOBAL_IMPLICIT & GLOBAL_EXPLICIT whereas 
pycodegen.CodeGenerator._nameOp seems not to know about GLOBAL_IMPLICIT/EXPLICIT 
but has only a single GLOBAL scope.
-- 
Robin Becker