[Python-Dev] Coverity Scan

[Terry Reedy]

On 7/25/2013 2:48 PM, Christian Heimes wrote:
> Hello,
>
> this is an update on my work and the current status of Coverity Scan.

Great work.

>
> Maybe you have noticed a checkins made be me that end with the line "CID
> #". These are checkins that fix an issue that was discovered by the
> static code analyzer Coverity. Coverity is a commercial product but it's
> a free service for some Open Source projects. Python has been analyzed
> by Coverity since about 2007. Guido, Neal, Brett, Stefan and some other
> developers have used Coverity before I took over. I fixed a couple of
> issues before 3.3 reached the RC phase and more bugs in the last couple
> of months.

The benefit for us is not just improving Python having external 
verification of its excellence in relation both to other open-source 
projects and commercial software.

> Coverity is really great and its web GUI is fun to use, too. I was able
> to identify and fix resource leaks, NULL pointer issues, buffer
> overflows and missing checks all over the place. Because it's a static
> analyzer that follows data-flows and control-flows the tool can detect
> issues in error paths that are hardly visited at all. I have started to
> document Coverity here:
>
>    http://docs.python.org/devguide/coverity.html
>
>
> Interview
> ---------
>
> A week ago I was contacted by Coverity. They have started a series of
> articles and press releases about Open Source projects that use their
> free service Coverity Scan, see
>
> http://www.coverity.com/company/press-releases/read/coverity-introduces-monthly-spotlight-series-for-coverity-scan-open-source-projects

The intention is to promote the best of open source to industry.

> Two days ago I had a lovely phone interview about my involvement in the
> Python project and our development style. They are going to release a
> nice article in a couple of weeks. In the mean time we have time to fix
> the remaining couple issues. We *might* be able to reach the highest
> coverity integrity level! I have dealt with all major issues so we just
> have to fix a couple of issues.


> Current stats
> -------------
>
> Lines of Code:	396,179

C only? or does Python code now count as 'source code'?

> Defect Density:	0.05

= defects per thousand lines = 20/400

Anything under 1 is good. The release above reports Samba now at .6.
http://www.pcworld.com/article/2038244/linux-code-is-the-benchmark-of-quality-study-concludes.html
reports Linux 3.8 as having the same for 7.6 million lines.

> Total defects:	1,054
> Outstanding:	   21 (Coverity Connect shows less)
> Dismissed:	  222

This implies that they accept our designation of some things as False 
Positives or Intentional. Does Coverity do any review of such 
designations, so a project cannot cheat?

> Fixed:		  811
>
> http://i.imgur.com/NoELjcj.jpg
> http://i.imgur.com/eJSzTUX.jpg
>
>
> open issues
> -----------
>
> http://bugs.python.org/issue17899
> http://bugs.python.org/issue18556
> http://bugs.python.org/issue18555
> http://bugs.python.org/issue18552
> http://bugs.python.org/issue18551
> http://bugs.python.org/issue18550
> http://bugs.python.org/issue18528

-- 
Terry Jan Reedy