[Python-Dev] Coverity Scan

[Terry Reedy]

On 7/25/2013 6:00 PM, Terry Reedy wrote:

>> Defect Density:    0.05
>
> = defects per thousand lines = 20/400
>
> Anything under 1 is good. The release above reports Samba now at .6.
> http://www.pcworld.com/article/2038244/linux-code-is-the-benchmark-of-quality-study-concludes.html
>
> reports Linux 3.8 as having the same for 7.6 million lines.
>
>> Total defects:    1,054
>> Outstanding:       21 (Coverity Connect shows less)
>> Dismissed:      222
>
> This implies that they accept our designation of some things as False
> Positives or Intentional. Does Coverity do any review of such
> designations, so a project cannot cheat?

I found the answer here
https://docs.google.com/file/d/0B5wQCOK_TiRiMWVqQ0xPaDEzbkU/edit
Coverity Integrity Level 1 is 1 (defect/1000 lines)
Level 2 is .1 (we have passed that)
Level 3 is .01 + no major defects + <20% (all all defects?) false 
positives as that is their normal rate.#

A higher false positive rates requires auditing by Coverity. They claim 
"A higher false positive rate indicates misconfiguration, usage of 
unusual idioms, or incorrect diagnosis of a large number of defects." 
They else add "or a flaw in our analysis."

# Since false positives should stay constant as true positives are 
reduced toward 0, false / all should tend toward 1 (100%) if I 
understand the ratio correctly.

>
>> Fixed:          811
>>
>> http://i.imgur.com/NoELjcj.jpg
>> http://i.imgur.com/eJSzTUX.jpg

-- 
Terry Jan Reedy