[issue39458] Multiprocessing.Pool maxtasksperchild=1 doesn't work
New submission from Gabriel Tardif <gabriel@zeroinfo.tech>: Hello This bug is about the maxtasksperchild parameter in the Pool object constructor of the multiprocessing module. When you set processes = 1 in the Pool constructor maxtasksperchild value is double by two for unknow raison whatever the maxtaskperchild value. As mentionned in the documentation, once the process has reach the maxtasksperchil value it should rebuild itself in the memory from the parent process. In the short python exemple provided below, you can see the value of showedFiles of each process incresing over 1 which is not normal if Pool constructor is set to processes = 1, maxtasksperchil = 1. The only running process should destroy / reset itself and so set its value 'showedFiles' to 0 first and 1 for each os.listdir() entry. ---------- assignee: docs@python components: Documentation files: reader.py messages: 360736 nosy: Gabriel Tardif, docs@python priority: normal severity: normal status: open title: Multiprocessing.Pool maxtasksperchild=1 doesn't work type: behavior versions: Python 3.6 Added file: https://bugs.python.org/file48864/reader.py _______________________________________ Python tracker <report@bugs.python.org> <https://bugs.python.org/issue39458> _______________________________________
Gabriel Tardif <gabriel@zeroinfo.tech> added the comment: Be aware that maxtasksperchild work together with the chunksize parameter of the map fonction when you use it and the default chunksize value is not 1, it's calculated according to your inputs, process and other parameters. ---------- resolution: -> not a bug _______________________________________ Python tracker <report@bugs.python.org> <https://bugs.python.org/issue39458> _______________________________________
Change by Gabriel Tardif <gabriel@zeroinfo.tech>: ---------- stage: -> resolved status: open -> closed _______________________________________ Python tracker <report@bugs.python.org> <https://bugs.python.org/issue39458> _______________________________________
participants (1)
-
Gabriel Tardif