[code-quality] Jedi gets a new parser

Claudiu Popa pcmanticore at gmail.com
Mon Sep 29 19:15:12 CEST 2014


On Mon, Sep 29, 2014 at 1:02 PM, Dave Halter <davidhalter88 at gmail.com> wrote:
> Hi all!
>
> I will be writing a new parser for Jedi. This is necessary for a quite a few
> reasons described in:
> https://github.com/davidhalter/jedi/issues/480.
>
> I'm posting this here, because some of you may be interested in actually
> using the new parser. It's going to be pure Python and contrary to the
> Python's internal parser will have error recovery and indent positions.
>
> If you're interested, I'd be really glad to hear your feedback!
>
> ~ Dave
>


Hello, Dave.

That's awesome, If your parser can generate a new AST, then we can
definitely use it, although we have
in mind to use lib2to3 to replace the standard ast module for our use
cases, but that can change if your parser
is finished until taking action from our side. Having always correct
information, such as line and col offsets definitely will
help us (we actually have a couple of problems regarding this, where
the lineno for some nodes is different than reality).
Regarding the parser approach, I'm not sure what's best in this situation.
I'm working on a static analysis tool, in the vein of pylint, for the
Powershell language
(https://github.com/RoPython/wispy) where I'm using modgrammar, a
recursive descent top down parser to parse the code
and retrieve an AST from it. modgrammar is pretty good, easy to use,
but slow sometimes and backtracks too much on ambiguities.
But it can memorize whitespace tokens and such, as long as you
explicitly define the whitespace token in the grammar, which
can be a pretty daunting task.

Anyway, if there is something I can help you with your parser, don't
hesitate to send me an email.


Claudiu


More information about the code-quality mailing list