[docs] [issue32666] Valgrind documentation seems to need updating

[Reuben Thomas]

New submission from Reuben Thomas <rrt at sc3d.org>:

I have just been trying to use Valgrind (in my case, to debug code in a library being tested via a Cython module).

It is working fine, but there seem to be some discrepancies between the documentation in README.valgrind and valgrind-python.supp on the one hand, and the actual code on the other.

README.Valgrind says (current git master):

Second, you must do one of the following:

  * Uncomment Py_USING_MEMORY_DEBUGGER in Objects/obmalloc.c,
    then rebuild Python
  * Uncomment the lines in Misc/valgrind-python.supp that
    suppress the warnings for PyObject_Free and PyObject_Realloc

However, Objects/objmalloc.c no longer contain this symbol (though Python/dtoa.c does).

Further, in https://bugs.python.org/issue31494 I find the following comment:

Also, I don't think --with-pydebug works well with valgrind.

It's either:
   1) --with-pydebug
   2) CFLAGS="-O0 -g" --with-valgrind
   3) CFLAGS="-O0 -g" --without-pymalloc

Combining 2) and 3) probably does not hurt, but is not necessary.

It would be useful to have this information in README.valgrind.

Also, the provided suppressions do not currently work with a normal build of Python, at least with Python 2.7 (with the PyObject_{Free,Realloc} suppressions uncommented). There seem to be at least two problems:

1. Extra symbols from LTO. This can be fixed by adding "*" to PyObject_Realloc in all its suppressions.

2. After fixing that, I still get one more "Invalid read of size 4":

==3227== Invalid read of size 4
==3227==    at 0x4FCEE2: long_dealloc.lto_priv.323 (abstract.c:2000)
==3227==    by 0x4BA22B: frame_dealloc (frameobject.c:458)
==3227==    by 0x4BA22B: PyEval_EvalCodeEx (ceval.c:3593)
==3227==    by 0x4C16E6: fast_function (ceval.c:4445)
==3227==    by 0x4C16E6: call_function (ceval.c:4370)

I can't see how this is related to any existing suppression.

I am using the stock Python 2.7 on Ubuntu 16.04. I find that I can work around both the above problems (that is, use the suppressions file as-is, just uncommenting the extra suppressions for PyObject_{Free,Realloc}) by using Ubuntu's debug build of Python (installed as /usr/bin/python-dbg). It is built with --with-pydebug.

Finally, README.valgrind says: "Valgrind is used periodically by Python developers to try to ensure there are no memory leaks or invalid memory reads/writes." Another reason to use Valgrind is, as in my case, to debug unrelated C code that is being run via Python (this is often a convenient way to write tests). I suggest deleting this sentence and making it more obvious how to simply make Python "transparent to Valgrind" for those trying to debug C libraries.

To summarise:

i. It seems that for current Python 3, this text:

  * Uncomment Py_USING_MEMORY_DEBUGGER in Objects/obmalloc.c,
    then rebuild Python

needs to be updated.

ii. For Python 2.7 at least, and possibly Python 3, extra suppressions are needed to use a standard build.

iii. It would be good to have the specific build advice from https://bugs.python.org/issue31494 in README.valgrind.

iv. It would be nice to clarify how to use a normal build of Python (e.g. by splitting the advice into "For Python developers" and "For Python users").

Given some guidance on what is correct, and what is desired/acceptable, I'd be happy to work up a patch to README.valgrind and valgrind-python.supp.

----------
assignee: docs at python
components: Documentation
messages: 310673
nosy: docs at python, rrt
priority: normal
severity: normal
status: open
title: Valgrind documentation seems to need updating
versions: Python 3.8

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue32666>
_______________________________________