[New-bugs-announce] [issue39672] SIGSEGV crash on shutdown with shelve & c pickle

[zd nex]

New submission from zd nex <zdnexnet at gmail.com>:

Hello,

so I was transferring some our old code from Python2.7 to new and find that new version seems to crash quite a lot. After some finding (good thing faulthandler) I think I tracked it down to to Shelve.__del__ method > going to C Pickle module (not python one). Here it is crash itself. Attached zip has 3 file. When shelve.close is used it does not seem to crash every time. 

$python3.8 -X faulthandler ce_test_2.py
start
end
Fatal Python error: Segmentation fault

Current thread 0x00007fb22e299740 (most recent call first):
  File "/usr/lib/python3.8/shelve.py", line 124 in __setitem__
  File "/usr/lib/python3.8/shelve.py", line 168 in sync
  File "/usr/lib/python3.8/shelve.py", line 144 in close
  File "/usr/lib/python3.8/shelve.py", line 162 in __del__
Neoprávněný přístup do paměti (SIGSEGV)


Code for crash is here:


import shelve
import material
data = shelve.open("test3", flag="c",writeback=True)

def test_shelve(data):
    for k,v in data.items():
        pass

print("start")
test_shelve(data)

#data.close() #fixes SIGSEGV at shutdown
#actually problem is in c pickle module; when Python pickle module is used it works

print("end")
#after this it is crash



Code just loads module and shelve and opens file. Then in another function it cycles through data and that creates crash in C pickle module at shutdown. Weird thing is that when cycle through data is not in function it does not crash. Also crash can be avoided when C Pickle is traded for Python Pickle.


In REPL it is quite similar just list on shelve.items() and exit makes Python crash.

Python 3.8.1 (default, Dec 22 2019, 08:15:39) 
[GCC 7.4.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import shelve
>>> import material
>>> data = shelve.open("test3", flag="c",writeback=True)
>>> list(data.items())
[('H1615', Material(name='Třešeň Romana', code='H1615', vars=0))]
>>> exit()
Fatal Python error: Segmentation fault

Current thread 0x00007f14a2546740 (most recent call first):
  File "/usr/lib/python3.8/shelve.py", line 124 in __setitem__
  File "/usr/lib/python3.8/shelve.py", line 168 in sync
  File "/usr/lib/python3.8/shelve.py", line 144 in close
  File "/usr/lib/python3.8/shelve.py", line 162 in __del__
Neoprávněný přístup do paměti (SIGSEGV)

Hopefully you can fix this.

----------
components: Library (Lib)
files: test_crash_shelve.zip
messages: 362186
nosy: zd nex
priority: normal
severity: normal
status: open
title: SIGSEGV crash on shutdown with shelve & c pickle
type: crash
versions: Python 3.6, Python 3.7, Python 3.8
Added file: https://bugs.python.org/file48899/test_crash_shelve.zip

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue39672>
_______________________________________