[code-quality] Jedi 0.9.0 is now a static analysis library

[Dave Halter]

Hi Kay

I will not write a very lengthy email here, but just guide you in a
general direction of how Jedi handles things.

First I want you to know that Jedi doesn't have a public API for its
parser, YET. This is intentional. As long as there are no real users
of it, why make it public? So if you're interested in a parser only
(not static analysis), RedBaron might be the better choice for now,
because Jedi's parser API is not offical yet. Pretty sure this will
change, though.

2015-05-03 11:40 GMT+02:00 Kay Hayen <kay.hayen at gmail.com>:
> With RedBaron, I can do this (bare with me on code quality, and by no means
> assume it's competent RedBaron usage):
>
> def updateString(string_node):
>     # Skip doc strings for now.
>     if string_node.parent.type in ("class", "def", None):
>         return
>
>     value = string_node.value
>
>     def isQuotedWith(quote):
>         return value.startswith(quote) and value.endswith(quote)
>
>     for quote in "'''", '"""', "'", '"':
>         if isQuotedWith(quote):
>             break
>     else:
>         assert False, value
>
>     real_value = value[len(quote):-len(quote)]
>     assert quote + real_value + quote == value
>
>     if "\n" not in real_value:
>         # Single characters, should be quoted with "'"
>         if len(eval(value)) == 1:
>             if real_value != "'":
>                 string_node.value = "'" + real_value + "'"
>         else:
>             if '"' not in real_value:
>                 string_node.value = '"' + real_value + '"'
>
> And then call this:
>
> for node in red.find_all("StringNode"):
>     try:
>         updateString(node)
>     except Exception:
>         print("Problem with", node)
>         node.help(deep = True, with_formatting = True)
>         raise
>
> It allows me do enforce some rules according to strings that are not
> multi-line. My rules there are, do not use triple quites without a new-line,
> strings of resulting length 1, become 'a' or '\n' and others, use "" or
> "ab", except of course "'" is valid and 'some "quote" would be too' as well.
>
> I do a bunch of these, and like to have these things, to e.g. make sure that
> my multi-line calling convention lines up the "=" signs nicely, etc,
> inserting and removing white space from tuples, comma separated stuff, etc.
>
> To me, it's not about, if I should do this, but if it can be done.

I think you could do that in a very similar way in Jedi. E.g. with this you can
get all docstrings:

    >>> for scope in p.module.walk():
    ...     print(repr(scope.raw_doc))
    ...
    ''

With this you get all string leafs:


    >>> from jedi.parser import tree
    >>> def nested(node):
    ...     for child in node.children:
    ...         if isinstance(node, tree.Node):
    ...             nested(node)
    ...         elif node.type == 'string':
    ...             print(node)  # You can change everything here.
    ... nested(jedi.parser.Parser(load_grammar(), SOURCE))

You can of course modify them. Look at the example below...


> So sure, I am asking of Jedi, if it has that bounding box, precisely to
> address this.
>
> With RedBaron, I can do this:
>
>>>> from redbaron import RedBaron
>>>> red = RedBaron("a+b")
>>>> red[0]
> a+b
>
>>>> red[0].second_formatting = "  "
>>>> red
> 0   a+  b
>
> That of course also means, it knows the "+" location, or I can infer it:
>
>>>> red[0].second.bounding_box
> BoundingBox (Position (1, 1), Position (1, 1))
>>>> red[0].first.bounding_box
> BoundingBox (Position (1, 1), Position (1, 1))
>>>> red[0].bounding_box
> BoundingBox (Position (1, 1), Position (1, 5))


    >>> from jedi.parser import Parser, load_grammar
    >>> Parser(load_grammar(), 'a + b')
    <Parser: <Module: None at 1-1>>
    >>> p = Parser(load_grammar(), 'a + b')
    >>> p.module.children
    [Node(simple_stmt, [Node(arith_expr, [<Name: a at 1,0>, <Operator:
+>, <Name: b at 1,4>]), <Whitespace: >]), <Whitespace: >]
    >>> expression = p.module.children[0].children[0]
    >>> expression
    Node(arith_expr, [<Name: a at 1,0>, <Operator: +>, <Name: b at 1,4>])
    >>> b = p.module.children[0].children[0].children[2]
    >>> b
    <Name: b at 1,4>
    >>> b.prefix
    ' '
    >>> b.start_pos
    (1, 4)
    >>> b.end_pos
    (1, 5)
    >>> expression.children[1]
    <Operator: +>
    >>> expression.children[1].start_pos
    (1, 2)
    >>> b.prefix = ''
    >>> b.value = '3' # Make the expression to '+3' instead of '+ b'
    >>> p.module.get_code()
    'a +3'


So as you can see, Jedi is all about positions. Jedi knows the positions of all
leafs as well as the prefixes (whitespace + comments) of that stuff.


> I would love for you to provide a code example how to use Jedi for editing
> like I did above, and if you think that creating such reports could be based
> on Jedi parsing.

I hope my example above is good enough. It shows how easy it is to play with
code. Jedi nodes are really simple, but still powerful. Let me know if
something is still unclear.

~ Dave