[New-bugs-announce] [issue35715] ProcessPool workers hold onto return value of last task in memory

[David Chevell]

New submission from David Chevell <chevell at gmail.com>:

ProcessPoolExecutor workers will hold onto the return value of their last task in memory until the next task is received. Since the return value has already been propagated to the parent process's `Future` or else effectively discarded, this is holding onto objects unnecessarily.

Simple case to reproduce:

    import concurrent.futures
    import time

    executor = concurrent.futures.ProcessPoolExecutor(max_workers=1)

    def big_val():
        return [{1:1} for i in range(1, 1000000)]

    executor.submit(big_val)

    # Observe the memory usage of the process worker during the sleep interval
    time.sleep(10)


This should be easily fixed by having the worker explicitly `del r` after calling `_sendback_result` as it already does this for `call_item`

----------
components: Library (Lib)
messages: 333444
nosy: dchevell
priority: normal
severity: normal
status: open
title: ProcessPool workers hold onto return value of last task in memory
versions: Python 3.8

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue35715>
_______________________________________