[code-quality] Static checker for common Python programming errors

[Dave Halter]

2014-11-25 12:29 GMT+01:00 Stefan Bucur <stefan.bucur at gmail.com>:

>
>
> On Tue Nov 18 2014 at 2:01:46 PM Dave Halter <davidhalter88 at gmail.com>
> wrote:
>
>> Hi Stefan
>>
>> I'm playing with this as well in Jedi. I'm pretty far with flow analysis
>> and AttributeErrors. (This includes everything you mention above except
>> integer division by zero). Would be easy to implement in Jedi, though. I
>> just have different priorities, at the moment.
>>
>> If you have some time on your hands you can watch my EuroPython talk
>> about this: https://www.youtube.com/watch?v=DfVHSw0iOsk I'm also glad to
>> skype (gaukler_) if you're interested. Jedi is not well known for doing
>> static analysis. But it's my goal to change this now.
>>
>
> Thanks Dave for the pointer. I watched your talk and had a look at Jedi's
> code — this is quite nice and would indeed be great to better expose the
> static analysis potential of the framework.
>
> What I'm trying to achieve is something a bit different. My goal is to
> reuse as much as possible the "implicit specs" of the interpreter itself
> (CPython), as opposed to implementing them again in my analysis tool.
>
> We already have an execution engine that uses the interpreter to
> automatically explore multiple paths through a piece of Python code. You
> can read here the academic paper, with case studies for Python and Lua:
> http://dslab.epfl.ch/pubs/chef.pdf
>

Stefan, I haven't read the full paper (but most of it). Interesting stuff!
Are you doing any "argument propagation" or are you simply looking at a
function without argument knowledge?

How do you does your output look like? Can you give us a real world example
(maybe simplejson)? Would be really interesting.

BTW: I remember asking the same question that you asked here in the
beginning: "What are your most typical mistakes that you would like a
static analysis tool pick up?"
https://github.com/davidhalter/jedi/issues/408 The first maybe 10 answers
might interest you.

~ Dave


> I'm now working on extending this technique to other types of analyses and
> I'm trying to determine the most relevant types of analyses and checks for
> Python. So far, I conclude that type inference would serve multiple
> purposes and is on the top of my list.
>
> Stefan
>
>
>> 2014-11-17 18:18 GMT+01:00 Stefan Bucur <stefan.bucur at gmail.com>:
>>
>>> I'm developing a Python static analysis tool that flags common
>>> programming errors in Python programs. The tool is meant to complement
>>> other tools like Pylint (which perform checks at lexical and AST level) by
>>> going deeper with the code analysis and keeping track of the possible
>>> control flow paths in the program (path-sensitive analysis).
>>>
>>> For instance, a path-sensitive analysis detects that the following
>>> snippet of code would raise an AttributeError exception:
>>>
>>> if object is None: # If the True branch is taken, we know the object is
>>> None
>>>   object.doSomething() # ... so this statement would always fail
>>>
>>> I wanted first to tap into people's experience and get a sense of what
>>> common pitfalls in the language & its standard library such a static
>>> checker should look for. Just as an example of what I mean, here [1] is a
>>> list of static checks for the C++ language, as part of the Clang static
>>> analyzer project.
>>>
>>> My preliminary list of Python checks is quite rudimentary, but maybe
>>> could serve as a discussion starter:
>>>
>>> * Proper Unicode handling (for 2.x)
>>>   - encode() is not called on str object
>>>   - decode() is not called on unicode object
>>> * Check for integer division by zero
>>> * Check for None object dereferences
>>>
>>> Thanks a lot,
>>> Stefan Bucur
>>>
>>> [1] http://clang-analyzer.llvm.org/available_checks.html
>>>
>>>
>>> _______________________________________________
>>> code-quality mailing list
>>> code-quality at python.org
>>> https://mail.python.org/mailman/listinfo/code-quality
>>>
>>>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/code-quality/attachments/20141125/51e4cd4f/attachment-0001.html>