[code-quality] The union of all tools

Carl Crowder carl.crowder at gmail.com
Mon Dec 8 22:34:46 CET 2014


Hi Skip,

This kind of argument was on my mind a little when I made prospector
(https://github.com/landscapeio/prospector) although probably from a
different angle. There are many tools out there and using them all
means you can get the maximal coverage of all concepts, with some
overlap. Each tool has a spectrum of certainty and usefulness, and
what you really want is the biggest problems in a specific code base.
"Biggest problem" as a concept varies from codebase to codebase.

Of course the ideal tool figures out exactly what you want and what
level of scrutiny you want on your code, and delivers the most
important problems first. Such a tool is impossible! I think you can
get there though, if you merge each tool (and therefore, each opinion
of what is important) enough.

The main concern is how to filter the wheat from the chaff. So far I
have simply made opinionated defaults and there is a way to make your
own configuration; that level of effort isn't that different to simply
configuring each tool yourself!

I'd love some feedback on this concept in general. I think each tool
and each error and each codebase and each user is a unique point and
it's difficult to create something that caters to all tastes. However,
having several starting positions tailored to individual use cases
could be helpful.

Cheers,
Carl

On 8 December 2014 at 22:10, Skip Montanaro <skip.montanaro at gmail.com> wrote:
> Thinking about the discussion I (inadvertently) started about what sorts of
> issues should cause tools like pylint and pyflakes to emit messages, got me
> thinking about how I try to use these tools. I don't know what the current
> suite of all available static checking tools is, but let me just assume for
> the moment that we have three tools: pylint, pyflakes, and (the probably now
> defunct) pychecker. Each one has different goals, which naturally affects
> the messages it can possibly emit. For example, because pychecker imports
> the modules mentioned in the code it analyzes, it is able to build symbol
> tables for extension modules. OTOH, it could be slower, the user might
> suffer if importing modules had side effects, and to get the most out of it,
> you were tied to the platforms for which you had compiled extension modules.
> I don't think pylint actually imports the code it analyzes, so it can be
> more platform-independent, but not as comprehensive as pychecker when trying
> to emit warnings about cross-module issues. It can, however, identify other
> problematic constructs.
>
> As a user of these tools, recognizing that none will ever be "complete", I
> try to use multiple tools precisely so I can see the union of all messages.
> In fact, some while ago (at least a couple years, probably longer), I wrote
> a shell script wrapper which invokes both pychecker and pylint, presenting
> me with the sorted (and very lightly massaged) output. If pyflakes is so
> conservative in what it will emit that it tries to avoid all possible false
> positives, it's less useful to me, precisely because the odds are low that
> it will report problems the other available tools won't catch. That is:
>
>    pylint ∪ pychecker ≈ pylint ∪ pychecker ∪ pyflakes
>
> making the incremental benefit to me of using pyflakes small.
>
> This is not to say that Phil doesn't have excellent reasons for his
> approach, just that there is a non-trivial trade-off in that approach.
>
> Skip
>
>
> _______________________________________________
> code-quality mailing list
> code-quality at python.org
> https://mail.python.org/mailman/listinfo/code-quality
>


More information about the code-quality mailing list