[Python-Dev] Valgrind on 2.2.2

[Guido van Rossum]

> I ran valgrind 1.0.3 on Python 2.2.2.  Well, almost 2.2.2, it was from
> 12 Oct.  After 90+ minutes and over 512 MB of RAM, there are no
> new/major issues to report.
> 
> The complete valgrind log (231k) can be found here: 
>         http://www.metaslash.com/py/valgrind-2_2_2.txt
> 
> I've cleaned up the log (200k) to remove some of the uninteresting stuff:
>         http://www.metaslash.com/py/valgrind-clean-2_2_2.txt

I looked at this one only.

> There are a few small memory leaks.  I think most of the leaks are
> related to threads.  The report contains memory still in use as well
> as leaks.  To find memory leaks, search for ' lost ' without quotes.
> It's quite possible some of the memory still in use is due to
> reference leaks.

This seems to be the most worrysome:

==28827== 268980 bytes in 15 blocks are possibly lost in loss record 100 of 106
==28827==    at 0x400487CD: realloc (vg_clientfuncs.c:270)
==28827==    by 0x80994DC: _PyObject_GC_Resize (Modules/gcmodule.c:917)
==28827==    by 0x80BD46E: PyFrame_New (Objects/frameobject.c:264)
==28827==    by 0x80782C3: PyEval_EvalCodeEx (Python/ceval.c:2390)
==28827==    by 0x807AA43: fast_function (Python/ceval.c:3173)
==28827==    by 0x80779B5: eval_frame (Python/ceval.c:2034)
==28827==    by 0x807892B: PyEval_EvalCodeEx (Python/ceval.c:2595)
==28827==    by 0x807AA43: fast_function (Python/ceval.c:3173)
==28827==    by 0x80779B5: eval_frame (Python/ceval.c:2034)
==28827==    by 0x807892B: PyEval_EvalCodeEx (Python/ceval.c:2595)

There are a few other records mentioning GC_Resize, but this one is
the biggest.  Could it be that the free frame list is botched?  OTOH,
what does "possibly lost" really mean?

There are also a few fingers pointing in the direction of weakref_ref,
e.g.

==28827== 520 bytes in 14 blocks are possibly lost in loss record 48 of 106
==28827==    at 0x400481B4: malloc (vg_clientfuncs.c:100)
==28827==    by 0x8099519: _PyObject_GC_New (Modules/gcmodule.c:868)
==28827==    by 0x8067BA5: PyWeakref_NewRef (Objects/weakrefobject.c:37)
==28827==    by 0x8066119: add_subclass (Objects/typeobject.c:2249)
==28827==    by 0x8061F29: PyType_Ready (Objects/typeobject.c:2219)
==28827==    by 0x80605A8: type_new (Objects/typeobject.c:1280)
==28827==    by 0x805EDA4: type_call (Objects/typeobject.c:183)
==28827==    by 0x80ABB0C: PyObject_Call (Objects/abstract.c:1688)
==28827==    by 0x807A34F: PyEval_CallObjectWithKeywords (Python/ceval.c:3058)
==28827==    by 0x80AB0C6: PyObject_CallFunction (Objects/abstract.c:1679)

Of course many of these could be caused by a single leak that drops a
pointer to a container -- then everything owned by that container is
also leaked.

I noticed this one:

==28713== 572 bytes in 15 blocks are possibly lost in loss record 39 of 78
==28713==    at 0x400481B4: malloc (vg_clientfuncs.c:100)
==28713==    by 0x8099519: _PyObject_GC_New (Modules/gcmodule.c:868)
==28713==    by 0x80B2D09: PyMethod_New (Objects/classobject.c:2008)
==28713==    by 0x80AF837: instance_getattr2 (Objects/classobject.c:702)
==28713==    by 0x80AF73A: instance_getattr1 (Objects/classobject.c:676)
==28713==    by 0x80B30F1: instance_getattr (Objects/classobject.c:715)
==28713==    by 0x80577A2: PyObject_GetAttr (Objects/object.c:1108)
==28713==    by 0x80B1731: half_cmp (Objects/classobject.c:1503)
==28713==    by 0x80B1937: instance_compare (Objects/classobject.c:1572)
==28713==    by 0x8055A6E: try_3way_compare (Objects/object.c:477)

which led me to an easy-to-fix leak in half_cmp(), both in 2.2.2 and 2.3:

diff -c -c -r2.154.8.1 classobject.c
*** classobject.c	13 Jun 2002 21:36:35 -0000	2.154.8.1
--- classobject.c	18 Oct 2002 00:36:06 -0000
***************
*** 1507,1514 ****
  	}
  
  	args = Py_BuildValue("(O)", w);
! 	if (args == NULL)
  		return -2;
  
  	result = PyEval_CallObject(cmp_func, args);
  	Py_DECREF(args);
--- 1507,1516 ----
  	}
  
  	args = Py_BuildValue("(O)", w);
! 	if (args == NULL) {
! 		Py_DECREF(cmp_func);
  		return -2;
+ 	}
  
  	result = PyEval_CallObject(cmp_func, args);
  	Py_DECREF(args);

but somehow I don't think that caused the report, because this exit
can only be taken if there's a memory error.  (Hm...  or if w == NULL
upon entry?  How could that happen?)

A similar one is on half_binop.

--Guido van Rossum (home page: http://www.python.org/~guido/)