[Python-Dev] How is obmalloc safe with "Invalid read of size 4" ?

[Karl Pickett]

We are having random, rare, nonreproducible segfaults/hangs with python2 on
ubuntu 14.04 in EC2.  I've managed to attach GDB to some hung ones and
there looks like clear memory corruption in the 'interned' hash table,
causing lookdict_string() to spin forever because all remaining slots have
a garbage 'key' pointer.  This happens just loading the 'site' module
dependencies, like 're' or 'codecs', before any of our code even gets run.

So we then tried running it under valgrind, and we got a lot of nasty
errors.  Even after reading the Misc/README.valgrind, which talks about
*uninitialized* reads being ok, I still don't see how reading from *freed*
memory would ever be safe, and why the suppression file thinks thats ok:

$ valgrind   ./pymd79/bin/python -c ""
==14651== Memcheck, a memory error detector
==14651== Copyright (C) 2002-2013, and GNU GPL'd, by Julian Seward et al.
==14651== Using Valgrind-3.9.0 and LibVEX; rerun with -h for copyright info
==14651== Command: ./pymd79/bin/python -c
==14651==
==14651== Invalid read of size 4
==14651==    at 0x461E40: Py_ADDRESS_IN_RANGE (obmalloc.c:1911)
==14651==    by 0x461EA3: PyObject_Free (obmalloc.c:994)
==14651==    by 0x4789AB: tupledealloc (tupleobject.c:235)
==14651==    by 0x5225BA: code_dealloc (codeobject.c:309)
==14651==    by 0x4CFFC3: load_source_module (import.c:1100)
==14651==    by 0x4D0E16: import_submodule (import.c:2700)
==14651==    by 0x4D1E19: PyImport_ImportModuleLevel (import.c:2515)
==14651==    by 0x4AE49A: builtin___import__ (bltinmodule.c:49)
==14651==    by 0x422C89: PyObject_Call (abstract.c:2529)
==14651==    by 0x4B12E5: PyEval_EvalFrameEx (ceval.c:3902)
==14651==    by 0x4B6A47: PyEval_EvalCodeEx (ceval.c:3265)
==14651==    by 0x4B6B71: PyEval_EvalCode (ceval.c:667)
==14651==  Address 0x5bcd020 is 2,256 bytes inside a block of size 2,801
free'd
==14651==    at 0x4C28577: free (in
/usr/lib64/valgrind/vgpreload_memcheck-amd64-linux.so)
==14651==    by 0x4DB2B0: PyMarshal_ReadLastObjectFromFile (marshal.c:1145)
==14651==    by 0x4CFE71: load_source_module (import.c:801)

- Karl
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20150324/76115737/attachment.html>