[code-quality] Code-quality & PEP257

[David Malcolm]

On Wed, 2013-04-17 at 15:31 +0200, Vladimir Keleshev wrote:
> Hi Ian,
>  
> I'm ready to assist you with integrating pep257 into flake8. Do you
> plan to vendorize it or make it a dependency? `Error` is the right
> class to look for. `__str__` formats the error. The default format
> looks like:
>  
>     path/file.py:14:4: PEP257 Class docstrings should have 1 blank
> line around them.
>  
> which looks pretty much the same as flake8's:
>  
>     coolproject/mod.py:97:1: F401 'shutil' imported but unused
>  
> except for dot in the end and no error/warning code. `check_source`
> will return a list of all errors in a string.
>  
> * * *
>  
> But still, as I was saying:
>  
> > Wouldn't having a
> > common error format between all the tools defeat the purpose of
> flake8?
>  
> About Firehose: I find it over-engineered. It is radically different
> from any current error formats. I think that only something simple can
> succeed when it comes to standards that cannot be enforced.

Hi!

I'm the creator of Firehose, so I clearly have a bias here :)

I guess the reason for the perceived complexity is that Firehose does a
lot of things:
* as well as capturing just an issue at a location in a source file, it
can also capture a trace of execution to reach the issue.  This data is
emitter by tools like clang-analyzer and my cpychecker tool, describing
a path through the source code that leads to a given problem.  (I
suspect that the current tools in flake8 don't do that level of data
output, so perhaps this complexity is redundant for your use-case)
* as well as capturing "issues" in the software under test (SUT),
Firehose can catch "failures" meaning a problem with the testing
software (e.g. a traceback, potentially associated with the line of code
in the SUT that broke the checker): if you have any of these, you don't
have full coverage of the software-under-test.   I've found this *very*
useful when doing mass-runs of my checker on large collections of code:
you can record the failures and come back to them later
* there's also the concept of "info" about the SUT: something that isn't
an "issue" per se, just data.  I use this for capturing McCabe
complexity of functions in C extension modules in cpychecker.
* potentially you may want to add CWE ID for errors, which in Firehose
is an optional field of a firehose.model.Issue.  cwe.mitre.org is a
sister project to CVE, attempting to classify the underlying kinds of
coding mistake that lead to security issues (e.g. "CWE-89" which means
"SQL Injection vulnerability", as opposed to CVE, which catalogs
specific instances of where a CWE occurred in a specific piece of code).

...and so on, this mail is probably too long already.

Also most of the firehose data model is optional.

But firehose isn't yet API-frozen (neither are the XML or JSON
serializations), so if you have ideas for improving things, let me know.

In any case, if you have a well-defined format, I'm sure we can write a
firehose.parsers module for slurping it into my data model, and a method
for getting it back out again.

Cheers; hope this is constructive
Dave

[...snip...]