[New-bugs-announce] [issue41577] Cannot use ProcessPoolExecutor if in a decorator?

Bob Fang report at bugs.python.org
Tue Aug 18 13:21:46 EDT 2020

New submission from Bob Fang <bob.fang.london at gmail.com>:

I have this minimal example:

from functools import wraps
from concurrent import futures
import random

def decorator(func):
    num_process = 4

    def impl(*args, **kwargs):
        with futures.ProcessPoolExecutor() as executor:
            fs = []
            for i in range(num_process):
                fut = executor.submit(func, *args, **kwargs)
            result = []
            for f in futures.as_completed(fs):
        return result
    return impl

def get_random_int():
    return random.randint(0, 100)

if __name__ == "__main__":
    result = get_random_int()
If we try to run this function I think we will have the following error:
_pickle.PicklingError: Can't pickle <function get_random_int at 0x7f06cee666a8>: it's not the same object as __main__.get_random_int
I think the main issue here is that the "wraps" decorator itself alters the `func` object and thus make it impossible to pickle. I found this rather strange. I am just wondering if there is any way to get around this behavior? I would want to use `wraps` if possible. Thanks!

components: Library (Lib)
messages: 375614
nosy: bob.fang.london
priority: normal
severity: normal
status: open
title: Cannot use ProcessPoolExecutor if in a decorator?
type: behavior
versions: Python 3.8

Python tracker <report at bugs.python.org>

More information about the New-bugs-announce mailing list