[New-bugs-announce] [issue3399] Memory corruption in multiprocessing module, OS X 10.5.4

[Mark Dickinson]

New submission from Mark Dickinson <dickinsm at gmail.com>:

As of revision 65077 of the trunk, I'm getting errors in 
test_multiprocessing that seem to point to memory corruption in object 
allocation/deallocation.  The failures are intermittent, and of a 
similar nature to the errors I was seeing previously, outlined in issue 
3088.

The platform is OS X 10.5.4 (not a fresh install---it was an upgrade 
from OS X 10.4, in case this makes any difference), running on a MacBook 
Pro.  I'm running a freshly checked out debug build of the trunk.

Here's what I did:

(1) make a fresh svn+ssh checkout of the trunk
(2) ./configure --with-pydebug && make
(3) ./python.exe Lib/test/test_multiprocessing.py
(4) repeat step (3) until something nasty happens.

The results vary from run to run, and 80-90% of the runs of 
test_multiprocessing pass.  Here are 3 of the failures I've seen, 
occurring on three separate runs of test_multiprocessing.

Failure 1:

test_notify_all (__main__.WithManagerTestCondition) ... Assertion 
failed: (pool->ref.count > 0), function PyObject_Free, file 
Objects/obmalloc.c, line 1100.

Failure 2:

test_imap_unordered (__main__.WithManagerTestPool) ...
python.exe(32381,0xb0513000) malloc: *** error for object 0xdbdbdbdb:
pointer being reallocated was not allocated
*** set a breakpoint in malloc_error_break to debug
python.exe(32381,0xb0513000) malloc: *** error for object 0xdbdbdbdb:
Non-aligned pointer being freed
*** set a breakpoint in malloc_error_break to debug
Fatal Python error: UNREF invalid object
ERROR

Failure 3:

test_imap_unordered (__main__.WithManagerTestPool) ... Fatal Python
error: UNREF invalid object
ERROR


I have very little (i.e. no) experience of debugging this kind of 
failure, and little understanding of how the multiprocessing module 
works.  But I can and will follow instructions and suggestions about how 
to debug this.

Stupid question:  it appears from reading the comments in that file that 
obmalloc.c is (intentionally) not thread-safe.  Could this have anything 
to do with the failures above?

----------
assignee: jnoller
components: Extension Modules
messages: 69917
nosy: jnoller, marketdickinson
severity: normal
status: open
title: Memory corruption in multiprocessing module, OS X 10.5.4
type: crash
versions: Python 2.6

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue3399>
_______________________________________