[New-bugs-announce] [issue47072] Database corruption with the shelve module

[Hubert Tournier]

New submission from Hubert Tournier <hubert.tournier at gmail.com>:

After adding a few records, the shelve module corrupts the database keys (the database is still readable if an element key is known, but no more iterable):

Traceback (most recent call last):
  File "./shelve-test.py", line 81, in <module>
    _verify_whois_cache()
  File "./shelve-test.py", line 61, in _verify_whois_cache
    for key in db.keys():
  File "/usr/local/lib/python3.8/_collections_abc.py", line 720, in __iter__
    yield from self._mapping
  File "/usr/local/lib/python3.8/shelve.py", line 95, in __iter__
    for k in self.dict.keys():
SystemError: Negative size passed to PyBytes_FromStringAndSize

I provide a short test program and data that systematically reproduces the bug. I added the a script showing execution messages, the resulting database in DB and text formats.

Tested with Python 3.8.12 on FreeBSD 13.0-RELEASE-p8.
I suppose Python is using my system package db5-5.3.28_8                   (Oracle Berkeley DB, revision 5.3).

See also similar issues:
https://bugs.python.org/issue33074
https://bugs.python.org/issue30388

----------
components: Library (Lib)
files: shelve-test.zip
messages: 415625
nosy: HubTou
priority: normal
severity: normal
status: open
title: Database corruption with the shelve module
type: behavior
versions: Python 3.8
Added file: https://bugs.python.org/file50693/shelve-test.zip

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue47072>
_______________________________________