I did a bit of research on a matter of different python interfaces to parsing. This is a bit of a mess, but it is as follows: a) there is a parser module which gives you very low-level interface (so-called concrete syntax tree), which is essentially a list of tuples with numbers inside. you can compile those tuples to bytecode afterwards. It's slowly being deprecated. b) there is a compiler module, which is a wrapper around parser. it does not let you compile ast, but it gives you ast. It's deprecated since 2.6. c) there is an ast module which lets you have ast *and* compile it. very neat. Now the state of the art is that we have parser, compiler is pure app-level and we don't provide ast. We already discussed a bit whether to have ast module for 1.1 and the answer is "very likely no", raise hands if you want to have it. Now I want to know how much do we care about all of this and how much do we care about any of those? My suggestions are as follows: 1. We keep parser as it is and don't try to fix bugs. Bugs (as reported by CPython's test failures) are that we don't do extra walk around the tree to detect syntax errors. those errors are only detected when we generated the code - or - 1a. We just always generate the code and throw it away. We loose performance, but we don't care (and we pass tests) 2. We slowly deprecate parser & compiler, with some removal in future 3. If we develop a new parser, we drop compatibility with exact concrete syntax trees and we only keep ast around (as ast module) if people want to interact with it (this is one less worry for writing parser). What do you think? Cheers, fijal
Maciej Fijalkowski wrote:
I did a bit of research on a matter of different python interfaces to parsing. This is a bit of a mess, but it is as follows:
[cut]
1a. We just always generate the code and throw it away. We loose performance, but we don't care (and we pass tests)
a quick google code search reveals that there are a couple of projects using parser, like pychecker, epydoc and quixote. If it's not too time consuming, I would go for this option.
2. We slowly deprecate parser & compiler, with some removal in future
+1
3. If we develop a new parser, we drop compatibility with exact concrete syntax trees and we only keep ast around (as ast module) if people want to interact with it (this is one less worry for writing parser).
That's an option, but not for 1.1, I think. ciao, Anto
participants (2)
-
Antonio Cuni
-
Maciej Fijalkowski