[Python-Dev] valgrind report

[Tim Peters]

[Neal Norwitz]

...

> Import/compile related memory leaks:
> ------------------------------------

These aren't actually related to anything <wink>.  The code starting at

>    by 0x807CDB7: new_arena (obmalloc.c:500)

is this:

		p = (uptr *)malloc(newmax * sizeof(*arenas));
		if (p == NULL)
			goto error;
		memcpy(p, arenas, narenas * sizeof(*arenas));
		arenas = p;	/* old arenas deliberately leaked */

Note the comment on the last line!  The reason for this is explained in
horrid detail in a comment block a few lines above this.  pymalloc maintains
a vector of pointers to arena base addresses, and when this vector itself
needs to grow, the memory it used to occupy is allowed to leak.  In the
absence of introducing expensive thread locks, we can't know whether some
other thread may still be referencing the old version of this vector (it's
OK if it does), so we can't free() it.  Any valgrind complaint with

>    by 0x807CDB7: new_arena (obmalloc.c:500)

in the listing is due to this deliberate leak, and the code lower in the
listing is irrelevant.  It just had the bad luck to call pymalloc exactly
when available space in all currently allocated arenas is in use, *and*
there's no space left in pymalloc's exponentially-overallocated vector of
arena base addresses to hold one more arena base address.

...

> Initialization memory leaks: ----------------------------
>
>  64 bytes in 1 blocks are definitely lost in loss record 139 of 663
>     at 0x3C01DA1D: malloc (vg_replace_malloc.c:109)
>     by 0x807CE32: new_arena (obmalloc.c:452)
>     by 0x807CAF2: PyObject_Malloc (obmalloc.c:699)
>     by 0x80E1566: _PyObject_GC_Malloc (gcmodule.c:1183)
>     by 0x80E166C: _PyObject_GC_NewVar (gcmodule.c:1214)
>     by 0x8084A71: PyTuple_New (tupleobject.c:68)
>     by 0x8087B2F: PyType_Ready (typeobject.c:3161)
>     by 0x8087B39: PyType_Ready (typeobject.c:3147)
>     by 0x807BE01: _Py_ReadyTypes (object.c:1802)
>     by 0x80D9B93: Py_Initialize (pythonrun.c:167)
>     by 0x805510A: Py_Main (main.c:376)
>     by 0x8054DAA: main (python.c:23)

That's really the same thing:  this is the *first* time the vector of arena
base addresses got allocated.  That memory is eventually allowed to leak by
the C code reproduced above.

This isn't worth any convolution "to fix".  Each pointer we save in the
vector of arena base addresses controls 256KB of memory, so it's an
insignificant space overhead.  The vector doubles in size whenever growth is
necessary, so the total amount of leaked space is approximately equal to the
current size of the vector.