Move encoding_decl to the top of Grammar/Grammar?
Hi all, Currently, Parser/parsetok.c has a dependency on graminit.h. This can cause headaches when rebuilding after adding new syntax to Grammar/Grammar because parsetok.c is part of pgen, which is responsible for *generating* graminit.h. This circular dependency can result in parsetok.c using a different value for encoding_decl to what is used in ast.c, which causes PyAST_FromNode to fall over at runtime. It effectively looks something like this: * Grammar/Grammar is modified * build begins -- pgen compiles, parsetok.c uses encoding_decl=X * graminit.h is rebuilt with encoding_decl=Y * ast.c is compiled using encoding_decl=Y * when python runs, parsetok() emits encoding_decl nodes that PyAST_FromNode can't recognize: SystemError: invalid node XXX for PyAST_FromNode A nice, easy short term solution that doesn't require unwinding this dependency would be to simply move encoding_decl to the top of Grammar/Grammar and add a big warning noting that it needs to come before everything else. This will help to ensure its value never changes when syntax is added/removed. I'm happy to provide a patch for this (including some additional dependency info for files dependent upon graminit.h and Python-ast.h), but was wondering if there were any opinions about how this should be resolved. Cheers, Tom
Here's the corresponding tracker issue: http://bugs.python.org/issue4347 I've uploaded a patch there anyway, since I'm going to need this stuff working for a presentation I'm giving tomorrow. Cheers, T Thomas Lee wrote:
Hi all,
Currently, Parser/parsetok.c has a dependency on graminit.h. This can cause headaches when rebuilding after adding new syntax to Grammar/Grammar because parsetok.c is part of pgen, which is responsible for *generating* graminit.h.
This circular dependency can result in parsetok.c using a different value for encoding_decl to what is used in ast.c, which causes PyAST_FromNode to fall over at runtime. It effectively looks something like this:
* Grammar/Grammar is modified * build begins -- pgen compiles, parsetok.c uses encoding_decl=X * graminit.h is rebuilt with encoding_decl=Y * ast.c is compiled using encoding_decl=Y * when python runs, parsetok() emits encoding_decl nodes that PyAST_FromNode can't recognize:
SystemError: invalid node XXX for PyAST_FromNode
A nice, easy short term solution that doesn't require unwinding this dependency would be to simply move encoding_decl to the top of Grammar/Grammar and add a big warning noting that it needs to come before everything else. This will help to ensure its value never changes when syntax is added/removed.
I'm happy to provide a patch for this (including some additional dependency info for files dependent upon graminit.h and Python-ast.h), but was wondering if there were any opinions about how this should be resolved.
Cheers, Tom _______________________________________________ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/tom%40vector-seven.com
participants (1)
-
Thomas Lee