[New-bugs-announce] [issue39492] reference cycle affecting Pickler instances (Python3.8+)

Pierre Glaser report at bugs.python.org
Wed Jan 29 18:39:28 EST 2020

New submission from Pierre Glaser <pierreglaser at msn.com>:

The new Pickler reducer_override mechanism introduced in `Python3.8` generates a reference cycle: for optimization purposes, a the pickler.reducer_override bound method is referenced into the reducer_override attribute of the Pickler's struct. Thus, until as a gc.collect call is performed, both the Pickler and all the elements it pickled (as they are part of its memo), wont be collected.

We should break this cycle a the end of the dump() method.

See reproducer below:
import threading
import weakref
import pickle
import io

class MyClass:

my_object = MyClass()
collect = threading.Event()
_ = weakref.ref(my_object, lambda obj: collect.set())  # noqa

class MyPickler(pickle.Pickler):
    def reducer_override(self, obj):
        return NotImplemented

my_pickler = MyPickler(io.BytesIO())
del my_object
del my_pickler

# import gc
# gc.collect()

for i in range(5):
    collected = collect.wait(timeout=0.1)
    if collected:
        print('my_object was successfully collected')

components: Library (Lib)
messages: 360995
nosy: pierreglaser, pitrou
priority: normal
severity: normal
status: open
title: reference cycle affecting Pickler instances (Python3.8+)
type: resource usage
versions: Python 3.8, Python 3.9

Python tracker <report at bugs.python.org>

More information about the New-bugs-announce mailing list