[New-bugs-announce] [issue43802] Seg fault on macOS using multiprocessing.JoinableQueue
Jacob Walls
report at bugs.python.org
Sat Apr 10 12:17:02 EDT 2021
New submission from Jacob Walls <jacobtylerwalls at gmail.com>:
macOS 10.13.6
Python 3.9.2
I can consistently reproduce a seg fault while using multiprocessing.JoinableQueue in Python 3.9.2.
My use case is the sheet music processing library music21. My fork includes a folder of 209 files I use to reproduce, running 3 cores, shown in the script below. (This is a subset of the over 1,000 files found here: https://github.com/MarkGotham/When-in-Rome/tree/master/Corpus/OpenScore-LiederCorpus
Using this set of 1,000 files reproduces nearly every time; using the 209 files I committed to my fork was enough to reproduce about 75% of the time.)
I'm a contributor to music21, so if this is an overwhelming amount of information to debug, I will gladly pare this down as much as I can or create some methods to access the multiprocessing functionality more directly. Many thanks for any assistance.
pip3 install git+https://github.com/jacobtylerwalls/music21.git@bpo-investigation
from music21 import corpus
# suggest using a unique name each attempt
lc = corpus.corpora.LocalCorpus(name='bpo-investigation')
# point to the directory of files I committed to my fork for this investigation
lc.addPath('/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/site-packages/music21/bpo-files')
# parse the files using multiprocessing
# calls music21.metadata.bundles.MetadataBundle.addFromPaths()
# which calls music21.metadata.caching.process_parallel()
lc.save()
# CTRL-C to recover from seg fault
# then, wipe out the entries in .music21rc so that you can cleanly reproduce again
from music21 import environment
us = environment.UserSettings()
us['localCorporaSettings'] = {}
quit()
Process: Python [31677]
Path: /Library/Frameworks/Python.framework/Versions/3.9/Resources/Python.app/Contents/MacOS/Python
Identifier: Python
Version: 3.9.2 (3.9.2)
Code Type: X86-64 (Native)
Parent Process: Python [31674]
Responsible: Python [31677]
User ID: 501
Date/Time: 2021-04-10 11:21:19.294 -0400
OS Version: Mac OS X 10.13.6 (17G14042)
Report Version: 12
Anonymous UUID: E7B0208A-19D6-ABDF-B3EA-3910A56B3E72
Sleep/Wake UUID: C4B83F57-6AD1-469E-82AE-88214FAA6283
Time Awake Since Boot: 140000 seconds
Time Since Wake: 5900 seconds
System Integrity Protection: enabled
Crashed Thread: 0 Dispatch queue: com.apple.main-thread
Exception Type: EXC_BAD_ACCESS (SIGSEGV)
Exception Codes: KERN_INVALID_ADDRESS at 0x0000000100b3acd8
Exception Note: EXC_CORPSE_NOTIFY
Termination Signal: Segmentation fault: 11
Termination Reason: Namespace SIGNAL, Code 0xb
Terminating Process: exc handler [0]
VM Regions Near 0x100b3acd8:
-->
__TEXT 00000001068bb000-00000001068bc000 [ 4K] r-x/rwx SM=COW [/Library/Frameworks/Python.framework/Versions/3.9/Resources/Python.app/Contents/MacOS/Python]
Thread 0 Crashed:: Dispatch queue: com.apple.main-thread
0 org.python.python 0x0000000106944072 PyObject_RichCompare + 258
1 org.python.python 0x0000000106943e9b PyObject_RichCompareBool + 43
2 org.python.python 0x00000001069ce3c0 min_max + 624
3 org.python.python 0x0000000106940bab cfunction_call + 59
4 org.python.python 0x0000000106901cad _PyObject_MakeTpCall + 365
5 org.python.python 0x00000001069d865c call_function + 876
6 org.python.python 0x00000001069d5b8b _PyEval_EvalFrameDefault + 25371
7 org.python.python 0x0000000106902478 function_code_fastcall + 104
8 org.python.python 0x00000001069d85cc call_function + 732
9 org.python.python 0x00000001069d5ad2 _PyEval_EvalFrameDefault + 25186
10 org.python.python 0x00000001069d92c3 _PyEval_EvalCode + 2611
11 org.python.python 0x0000000106902401 _PyFunction_Vectorcall + 289
12 org.python.python 0x00000001069d85cc call_function + 732
13 org.python.python 0x00000001069d5ad2 _PyEval_EvalFrameDefault + 25186
14 org.python.python 0x00000001069d92c3 _PyEval_EvalCode + 2611
15 org.python.python 0x0000000106902401 _PyFunction_Vectorcall + 289
16 org.python.python 0x0000000106901b05 _PyObject_FastCallDictTstate + 293
17 org.python.python 0x00000001069026e8 _PyObject_Call_Prepend + 152
18 org.python.python 0x000000010695be85 slot_tp_init + 165
19 org.python.python 0x00000001069573d9 type_call + 345
20 org.python.python 0x0000000106901cad _PyObject_MakeTpCall + 365
21 org.python.python 0x00000001069d865c call_function + 876
22 org.python.python 0x00000001069d5af3 _PyEval_EvalFrameDefault + 25219
23 org.python.python 0x0000000106902478 function_code_fastcall + 104
24 org.python.python 0x00000001069044ba method_vectorcall + 202
25 org.python.python 0x00000001069d85cc call_function + 732
26 org.python.python 0x00000001069d5af3 _PyEval_EvalFrameDefault + 25219
27 org.python.python 0x0000000106902478 function_code_fastcall + 104
28 org.python.python 0x00000001069d85cc call_function + 732
29 org.python.python 0x00000001069d5ad2 _PyEval_EvalFrameDefault + 25186
30 org.python.python 0x0000000106902478 function_code_fastcall + 104
31 org.python.python 0x00000001069d85cc call_function + 732
32 org.python.python 0x00000001069d5ad2 _PyEval_EvalFrameDefault + 25186
33 org.python.python 0x0000000106902478 function_code_fastcall + 104
34 org.python.python 0x00000001069d85cc call_function + 732
35 org.python.python 0x00000001069d5ad2 _PyEval_EvalFrameDefault + 25186
36 org.python.python 0x00000001069d92c3 _PyEval_EvalCode + 2611
37 org.python.python 0x0000000106902401 _PyFunction_Vectorcall + 289
38 org.python.python 0x00000001069d85cc call_function + 732
39 org.python.python 0x00000001069d5ad2 _PyEval_EvalFrameDefault + 25186
40 org.python.python 0x0000000106902478 function_code_fastcall + 104
41 org.python.python 0x00000001069d85cc call_function + 732
42 org.python.python 0x00000001069d5b8b _PyEval_EvalFrameDefault + 25371
43 org.python.python 0x00000001069d92c3 _PyEval_EvalCode + 2611
44 org.python.python 0x0000000106902401 _PyFunction_Vectorcall + 289
45 org.python.python 0x00000001069d85cc call_function + 732
46 org.python.python 0x00000001069d5c21 _PyEval_EvalFrameDefault + 25521
47 org.python.python 0x00000001069d92c3 _PyEval_EvalCode + 2611
48 org.python.python 0x00000001069cf74b PyEval_EvalCode + 139
49 org.python.python 0x0000000106a21fc4 PyRun_StringFlags + 356
50 org.python.python 0x0000000106a21e15 PyRun_SimpleStringFlags + 69
51 org.python.python 0x0000000106a3e367 Py_RunMain + 1047
52 org.python.python 0x0000000106a3eaef pymain_main + 223
53 org.python.python 0x0000000106a3eceb Py_BytesMain + 43
54 libdyld.dylib 0x00007fff5a148015 start + 1
----------
components: macOS
messages: 390721
nosy: jacobtylerwalls, ned.deily, ronaldoussoren
priority: normal
severity: normal
status: open
title: Seg fault on macOS using multiprocessing.JoinableQueue
type: crash
versions: Python 3.9
_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue43802>
_______________________________________
More information about the New-bugs-announce
mailing list