[New-bugs-announce] [issue32421] Keeping an exception in cache can segfault the interpreter

[Yonatan Zunger]

New submission from Yonatan Zunger <zunger at gmail.com>:

Using the following decorator to memoize a function causes the interpreter to segfault on exit if any exceptions were stored in the cache. Changing the bad line to instead store copy.copy(e) makes the problem go away.

Almost certainly some kind of problem with the GC of the traceback not happening at the moment the interpreter expects it, but I haven't had time to investigate any more deeply yet.

def memoize(keyFunc=lambda x: x, cacheExceptions: bool=True):
    def wrapper(func):
        return _Memo(func, keyFunc, cacheExceptions)
    return wrapper

class _Memo(object):
    def __init__(self, func, keyFunc, cacheExceptions: bool) -> None:
        self.func = func
        self.keyFunc = keyFunc
        self.cacheExceptions = cacheExceptions
        self.lock = threading.Lock()
        self.cache = {}

    def __call__(self, *args, **kwargs) -> Any:
        key = self.keyFunc(*args, **kwargs)
        assert isinstance(key, collections.Hashable)
        with self.lock:
            if key in self.cache:
                value = self.cache[key]
                if isinstance(value, BaseException):
                    raise value
                return value

        try:
            result = self.func(*args, **kwargs)
        except BaseException as e:
            if self.cacheExceptions:
                with self.lock:
                    self.cache[key] = e   # BAD LINE
            six.reraise(*sys.exc_info())
        else:
            with self.lock:
                self.cache[key] = result
            return result

----------
components: Interpreter Core
messages: 308978
nosy: zunger
priority: normal
severity: normal
status: open
title: Keeping an exception in cache can segfault the interpreter
type: crash
versions: Python 3.6

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue32421>
_______________________________________