[Python-Dev] Move encoding_decl to the top of Grammar/Grammar?

[Thomas Lee]

Here's the corresponding tracker issue:

http://bugs.python.org/issue4347

I've uploaded a patch there anyway, since I'm going to need this stuff 
working for a presentation I'm giving tomorrow.

Cheers,
T

Thomas Lee wrote:
> Hi all,
>
> Currently, Parser/parsetok.c has a dependency on graminit.h. This can 
> cause headaches when rebuilding after adding new syntax  to 
> Grammar/Grammar because parsetok.c is part of pgen, which is 
> responsible for *generating* graminit.h.
>
> This circular dependency can result in parsetok.c using a different 
> value for encoding_decl to what is used in ast.c, which causes 
> PyAST_FromNode to fall over at runtime. It effectively looks something 
> like this:
>
> * Grammar/Grammar is modified
> * build begins -- pgen compiles, parsetok.c uses encoding_decl=X
> * graminit.h is rebuilt with encoding_decl=Y
> * ast.c is compiled using encoding_decl=Y
> * when python runs, parsetok() emits encoding_decl nodes that 
> PyAST_FromNode can't recognize:
>
> SystemError: invalid node XXX for PyAST_FromNode
>
> A nice, easy short term solution that doesn't require unwinding this 
> dependency would be to simply move encoding_decl to the top of 
> Grammar/Grammar and add a big warning noting that it needs to come 
> before everything else. This will help to ensure its value never 
> changes when syntax is added/removed.
>
> I'm happy to provide a patch for this (including some additional 
> dependency info for files dependent upon graminit.h and Python-ast.h), 
> but was wondering if there were any opinions about how this should be 
> resolved.
>
> Cheers,
> Tom
> _______________________________________________
> Python-Dev mailing list
> Python-Dev at python.org
> http://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe: 
> http://mail.python.org/mailman/options/python-dev/tom%40vector-seven.com