[issue33082] multiprocessing docs bury very important 'callback=' args
New submission from Chad <millchad@amazon.com>: Callbacks are really important in multiprocessing. Doc writer almost ignores them. ---------- assignee: docs@python components: Documentation messages: 313905 nosy: chadmiller-amzn, docs@python priority: normal severity: normal status: open title: multiprocessing docs bury very important 'callback=' args type: enhancement versions: Python 2.7, Python 3.4, Python 3.5, Python 3.6, Python 3.7, Python 3.8 _______________________________________ Python tracker <report@bugs.python.org> <https://bugs.python.org/issue33082> _______________________________________
Change by Ned Deily <nad@python.org>: ---------- nosy: +davin, pitrou versions: -Python 3.4, Python 3.5 _______________________________________ Python tracker <report@bugs.python.org> <https://bugs.python.org/issue33082> _______________________________________
Chad <millchad@amazon.com> added the comment: https://github.com/chadmiller-amzn/cpython/pull/1 (Putting that in "GitHub PR" field says "Edit Error: Unknown PR format, acceptable formats are: full github URL, #pr_number, pr_number") ---------- versions: +Python 3.4, Python 3.5 _______________________________________ Python tracker <report@bugs.python.org> <https://bugs.python.org/issue33082> _______________________________________
Mariatta Wijaya <mariatta.wijaya@gmail.com> added the comment: Can you make your PR in python/cpython GitHub repo, instead of your own fork? ---------- nosy: +Mariatta _______________________________________ Python tracker <report@bugs.python.org> <https://bugs.python.org/issue33082> _______________________________________
Change by Mariatta Wijaya <mariatta.wijaya@gmail.com>: ---------- versions: -Python 3.4, Python 3.5 _______________________________________ Python tracker <report@bugs.python.org> <https://bugs.python.org/issue33082> _______________________________________
Change by Chad <millchad@amazon.com>: ---------- keywords: +patch pull_requests: +5886 stage: -> patch review _______________________________________ Python tracker <report@bugs.python.org> <https://bugs.python.org/issue33082> _______________________________________
Antoine Pitrou <pitrou@free.fr> added the comment: I don't really agree that "callbacks are really important in multiprocessing" (I think I've hardly ever used them). I agree with the other doc changes in the PR. ---------- _______________________________________ Python tracker <report@bugs.python.org> <https://bugs.python.org/issue33082> _______________________________________
Chad <millchad@amazon.com> added the comment: On topic: My CLA is signed as of Monday, 19 March. My status here is not updated yet. pitrou, off-topic: Without callbacks, users who want multiprocessing functions to return something, not just mutate state somewhere else, must gather jobs in a list and continually iterate through them polling to see if each has finished yet and conditionally popping it from the list. It's expensive and ugly and error-prone. Callbacks are really great, you should try them again. So much better: pool.apply_async(func, args, callback=when_finished_call_with_result) ---------- _______________________________________ Python tracker <report@bugs.python.org> <https://bugs.python.org/issue33082> _______________________________________
Change by Chad <millchad@amazon.com>: ---------- pull_requests: +6150 _______________________________________ Python tracker <report@bugs.python.org> <https://bugs.python.org/issue33082> _______________________________________
Ned Deily <nad@python.org> added the comment: These doc changes have been languishing. We have a difference of opinion about adding a callback example. I don't have a strong feeling one way or the other. Davin or anyone else, what say you? ---------- nosy: +ned.deily _______________________________________ Python tracker <report@bugs.python.org> <https://bugs.python.org/issue33082> _______________________________________
Julien Palard <julien+python@palard.fr> added the comment: I agree Antoine on this one, if one want the result, It'll get it from the returned value (.get method in the example), or simply by using the not-async versions and directly get the results as a return value of the call. Also the given example in the documentation won't work as the result is never waited for, the pool gets destroyed before the sleep have the time to sleep. ---------- nosy: +mdk _______________________________________ Python tracker <report@bugs.python.org> <https://bugs.python.org/issue33082> _______________________________________
Julien Palard <julien+python@palard.fr> added the comment: I'd wait for Davin's review, but I'd keep the documentation as they are. ---------- _______________________________________ Python tracker <report@bugs.python.org> <https://bugs.python.org/issue33082> _______________________________________
Davin Potts <python@discontinuity.net> added the comment: I appreciate the functionality offered by the callbacks and have found good uses for them, as Chad clearly does/has. That said, the thought of expanding the documentation on the callbacks had not come up for me. Reading through the proposed changes to the prose explanations, the choice of words has changed but not significantly and virtually no new concepts are being explained. I agree with Julien that the docs should stay as they are. Chad: Thank you for advocating for things you think more people need to know about even if we do not update the docs this time. ---------- resolution: -> rejected stage: patch review -> resolved status: open -> closed _______________________________________ Python tracker <report@bugs.python.org> <https://bugs.python.org/issue33082> _______________________________________
participants (6)
-
Antoine Pitrou -
Chad
-
Davin Potts
-
Julien Palard
-
Mariatta Wijaya
-
Ned Deily