[Python-Dev] About "Coverity Study Ranks LAMP Code Quality"

[Neal Norwitz]

On 3/13/06, Greg Ewing <greg.ewing at canterbury.ac.nz> wrote:
> Fredrik Lundh wrote:
>
> > > But I'm wondering if the actual "bugs" list was transmitted to Python developers,
> > > and verified / acted upon.
> >
> > and in case it wasn't clear from my previous post, the answer to
> > your specific question is "yes" ;-)

16 people have access, 2-3 of which I haven't heard of.  6+ different
developers have reviewed at least some of the reports.

> Could whoever did this perhaps post a brief description
> of what sort of information their tool produces, and
> how good a hit rate they have of finding real problems?

I can post info, but it won't be brief. :-)

Their reports were high quality and accurate.  Of the false positives,
it was difficult for the tool to determine that the condition they are
checking really doesn't occur.  One example is doing a
PyString_Check() then using PyString_AsString() (or Size).  The tool
complained that they could return NULL/negative number, which is true
in general.  However, since we already know the object is a string
since we already verified it, we know those APIs can't return error
conditions.

Current results:
 Uninspected 12
 False Positives 13
 Ignore 24
 Resolved 3
 Total 51

For our purposes, False, Ignore, and Resolved are all pretty much the
same in that the warnings are no longer pertinent.  Actually, one of
the resolved is still outstanding as Jeff Epler? pointed out.

http://scan.coverity.com/ shows Python originally had 96 reported
issues, currently 15 are outstanding as of 2006-Mar-13 01:16:18
(presumably PST).  This does not include all C code checked in, since
their report includes code: 272,268, comment: 41,269, blank 38,496
(total: 352033) and:

$ wc -1 `find . -name '*.[ch]'` | tail -n 1
  425810

The public report says 15, but the current developer report shows 12. 
I'm not sure why there is a discrepancy.  All 12 are in ctypes which
was recently imported.

The reports provide information about what condition is violated and
where.  Usually this means there are at least 2, somtimes more,
annotations in the code to point out the originating condition and
problem area.  The types of problems reported in Python include: 
unintialized variables, resource leak, negative return values, using a
NULL pointer, dead code, use after free and some other similar
conditions.

Here's a report of the current disposition of issues:

Name  	 Uninspected  	 Bug  	 False  	 Ignore  	 Pending  	 Resolved  	 Total
<no project> 	0 	0 	0 	1 	0 	0 	1
Everything 	12 	0 	13 	23 	0 	3 	51
Modules 	12 	0 	2 	8 	0 	0 	22
Modules/expat 	0 	0 	1 	0 	0 	0 	1
Objects 	0 	0 	6 	8 	0 	1 	15
Parser 	0 	0 	0 	3 	0 	0 	3
Python 	0 	0 	5 	6 	0 	2 	13
Include 	0 	0 	0 	0 	0 	0 	0
Modules/cjkcodecs 	0 	0 	0 	0 	0 	0 	0
usr 	0 	0 	0 	0 	0 	0 	0
usr/X11R6 	0 	0 	0 	0 	0 	0 	0
usr/X11R6/include 	0 	0 	0 	0 	0 	0 	0
usr/include 	0 	0 	0 	0 	0 	0 	0
usr/include/arpa 	0 	0 	0 	0 	0 	0 	0
usr/include/asm 	0 	0 	0 	0 	0 	0 	0
usr/include/bits 	0 	0 	0 	0 	0 	0 	0
usr/include/bluetooth 	0 	0 	0 	0 	0 	0 	0
usr/include/et 	0 	0 	0 	0 	0 	0 	0
usr/include/gdbm 	0 	0 	0 	0 	0 	0 	0
usr/include/gnu 	0 	0 	0 	0 	0 	0 	0
usr/include/linux 	0 	0 	0 	0 	0 	0 	0
usr/include/ncurses 	0 	0 	0 	0 	0 	0 	0
usr/include/net 	0 	0 	0 	0 	0 	0 	0
usr/include/netinet 	0 	0 	0 	0 	0 	0 	0
usr/include/netpacket 	0 	0 	0 	0 	0 	0 	0
usr/include/openssl 	0 	0 	0 	0 	0 	0 	0
usr/include/readline 	0 	0 	0 	0 	0 	0 	0
usr/include/rpc 	0 	0 	0 	0 	0 	0 	0
usr/include/rpcsvc 	0 	0 	0 	0 	0 	0 	0
usr/include/sys 	0 	0 	0 	0 	0 	0 	0
usr/lib 	0 	0 	0 	0 	0 	0 	0
usr/lib/gcc 	0 	0 	0 	0 	0 	0 	0