Hi Skip, This kind of argument was on my mind a little when I made prospector (https://github.com/landscapeio/prospector) although probably from a different angle. There are many tools out there and using them all means you can get the maximal coverage of all concepts, with some overlap. Each tool has a spectrum of certainty and usefulness, and what you really want is the biggest problems in a specific code base. "Biggest problem" as a concept varies from codebase to codebase. Of course the ideal tool figures out exactly what you want and what level of scrutiny you want on your code, and delivers the most important problems first. Such a tool is impossible! I think you can get there though, if you merge each tool (and therefore, each opinion of what is important) enough. The main concern is how to filter the wheat from the chaff. So far I have simply made opinionated defaults and there is a way to make your own configuration; that level of effort isn't that different to simply configuring each tool yourself! I'd love some feedback on this concept in general. I think each tool and each error and each codebase and each user is a unique point and it's difficult to create something that caters to all tastes. However, having several starting positions tailored to individual use cases could be helpful. Cheers, Carl On 8 December 2014 at 22:10, Skip Montanaro <skip.montanaro@gmail.com> wrote:
Thinking about the discussion I (inadvertently) started about what sorts of issues should cause tools like pylint and pyflakes to emit messages, got me thinking about how I try to use these tools. I don't know what the current suite of all available static checking tools is, but let me just assume for the moment that we have three tools: pylint, pyflakes, and (the probably now defunct) pychecker. Each one has different goals, which naturally affects the messages it can possibly emit. For example, because pychecker imports the modules mentioned in the code it analyzes, it is able to build symbol tables for extension modules. OTOH, it could be slower, the user might suffer if importing modules had side effects, and to get the most out of it, you were tied to the platforms for which you had compiled extension modules. I don't think pylint actually imports the code it analyzes, so it can be more platform-independent, but not as comprehensive as pychecker when trying to emit warnings about cross-module issues. It can, however, identify other problematic constructs.
As a user of these tools, recognizing that none will ever be "complete", I try to use multiple tools precisely so I can see the union of all messages. In fact, some while ago (at least a couple years, probably longer), I wrote a shell script wrapper which invokes both pychecker and pylint, presenting me with the sorted (and very lightly massaged) output. If pyflakes is so conservative in what it will emit that it tries to avoid all possible false positives, it's less useful to me, precisely because the odds are low that it will report problems the other available tools won't catch. That is:
pylint ∪ pychecker ≈ pylint ∪ pychecker ∪ pyflakes
making the incremental benefit to me of using pyflakes small.
This is not to say that Phil doesn't have excellent reasons for his approach, just that there is a non-trivial trade-off in that approach.
Skip
_______________________________________________ code-quality mailing list code-quality@python.org https://mail.python.org/mailman/listinfo/code-quality