[code-quality] RedBaron, a bottom-up refactoring lib/tool for python

[Laurent Peuch]

Hello everyone and thanks for your answers :)

> > > I'm currently working on the integration of the lib2to3 parser into Jedi.
> > > This would make refactoring really easy (I'm about 50% done with the
> > > parser). It's also well tested and offers a few other advantages.
> > >
> > > In a perfect world, we could now combine our projects :-) I will look in
> > > detail at Red Baron on Monday.
> >
> > David, we've been talking about this during the latest EuroPython, and I've
> > talked with Laurent yesterday at the Capitole du Libre in Toulouse: IMO we
> > could
> > start by extracting from lib2to3 "the" parser that could be used by every
> > tools
> > like ours (refactoring, completion, static analysis...). It would be:
> > * loss-less (comments, indents...)
> > * accurate (eg from/to line numbers)
> > * fast
> > * version agnostic within a reasonable frame (eg 2.7 -> 3.4?)
> >
> 
> Yes, that's what I'm trying to create now (based on lib2to3). However my
> biggest problem is that I have to rewrite my evaluation engine as well,
> because it was depending on the old parser. I have two additional
> constraints:
> 
> - decent error recovery
> - memory efficient
> 
> The "fast" part is something I'm very eager to implement. I have done this
> before with my old parser. My approach is to re-parse only the parts of the
> file that have changed.
> My idea of "version agnostic" is to have the parser and the evaluation
> engine try to adhere to one version. My goal is to give Jedi a Python
> version number and Jedi would work according to that. This would make
> linting really cool, because you can run the linter for different Python
> versions.
> 
> I would recommend you to wait until I have finished my parser (1-2 months)
> or can at least report back. You can then either take the parser and fork
> it or take the evaluation engine as well.

Well, wouldn't it be a good idea if instead of us waiting 1-2 months
that you finish your parser then seeing if it also fits our needs we
discuss a bit of what each one of us needs and see if we could find an
agreement on what could be a common ST for all of us?

I can totally understand that discussing this right now might not be
the most appealing idea but maybe think a bit about it of the benefit
that we could get from a common AST for all of us:

* only one code base to maintain instead of everyone doing his
  own parser on his side
* we could share our efforts on making it as good as possible
* more time spend on doing the actual tools than the backend
* a de facto reference for every python developer that wants to join
  the field of tooling: more people -> more tools -> better python in
  general

I really think that we should at least try to see if it's possible,
having this kind of tool would really benefit us and the python
community in general.

What do you think?

-- 

Laurent Peuch -- Bram