[docs] [issue34172] multiprocessing.Pool and ThreadPool leak resources after being deleted

[STINNER Victor]

STINNER Victor <vstinner at redhat.com> added the comment:

tzickel:
> A. The documentation explicitly says: "When the pool object is garbage collected terminate() will be called immediately." (Happened till a code refactor 9 years ago introduced this bug).

It is a *very bad* practice to rely on __del__. Don't do that. That's why we introduced ResourceWarning.


tzickel:
> https://github.com/python/cpython/blob/master/Lib/multiprocessing/util.py#L147 (Till the end of the file almost)

Is this API *incompatible* with pool.close()? Explicitly release resources?


Pablo:
> Víctor, I have a PR fixing this in: (...) The fix is really simple, so I think we should fix it and not revert the change on this case.

I'm not comfortable with the fix. I cannot explain why but I feel like adding a strong dependency from a child to its parent is going to lead to more bugs, not less. It sounds like a recipe for reference cycles. Maybe I'm just plain wrong.

At this point, I would like that someone explains me what the problem is. https://github.com/python/cpython/pull/10852 is a solution, ok, but what is the problem? What does the code hangs, whereas previously it was fine? Is the example code really correct? Do we want to support such usage?

I understand that msg330864 rely on black magic to expect that it's going to be fine. The lifetime of the pool is implicit and it sounds like a bad design. I don't want to endorse that.

----------

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue34172>
_______________________________________