Claudiu, Peter Does anyone of you know why lib2to3 is using both a bottom-up parser and pgen2? It "looks" like there are two parsers. I'm quite confused by the parser design of lib2to3, because in older versions (python 2.6) there's only pgen2. ~ Dave 2014-09-29 19:15 GMT+02:00 Claudiu Popa <pcmanticore@gmail.com>:

On Mon, Sep 29, 2014 at 1:02 PM, Dave Halter <davidhalter88@gmail.com> wrote:

...

Hi all!

I will be writing a new parser for Jedi. This is necessary for a quite a few reasons described in: https://github.com/davidhalter/jedi/issues/480.

I'm posting this here, because some of you may be interested in actually using the new parser. It's going to be pure Python and contrary to the Python's internal parser will have error recovery and indent positions.

If you're interested, I'd be really glad to hear your feedback!

~ Dave

Hello, Dave.

That's awesome, If your parser can generate a new AST, then we can definitely use it, although we have in mind to use lib2to3 to replace the standard ast module for our use cases, but that can change if your parser is finished until taking action from our side. Having always correct information, such as line and col offsets definitely will help us (we actually have a couple of problems regarding this, where the lineno for some nodes is different than reality). Regarding the parser approach, I'm not sure what's best in this situation. I'm working on a static analysis tool, in the vein of pylint, for the Powershell language (https://github.com/RoPython/wispy) where I'm using modgrammar, a recursive descent top down parser to parse the code and retrieve an AST from it. modgrammar is pretty good, easy to use, but slow sometimes and backtracks too much on ambiguities. But it can memorize whitespace tokens and such, as long as you explicitly define the whitespace token in the grammar, which can be a pretty daunting task.

Anyway, if there is something I can help you with your parser, don't hesitate to send me an email.

Claudiu