[Python-Dev] bpo-34837: Multiprocessing.Pool API Extension - Pass Data to Workers w/o Globals

[Sean Harrington]

I am proposing an extension to the multiprocessing.Pool API that allows for
an alternative way to pass data to Pool worker processes, *without* using
globals.

A PR has been opened <https://github.com/python/cpython/pull/9627>,
extensive test coverage is also included, with all tests & CI passing on
github.

Please see this blog post
<https://thelaziestprogrammer.com/python/multiprocessing-pool-expect-initret-proposal>
for details, motivation, and use cases of the API extension before reading
on.

In *short*, the implementation of the feature works as follows:

   1. Exposes a kwarg on Pool.__init__ called `expect_initret`, that
   defaults to False. When set to True:
      1. Capture the return value of the initializer kwarg of Pool
      2. Pass this value to the function being applied, as a kwarg.

Again, in *short,* the motivation of the feature is to provide an explicit
"flow of data" from parent process to worker process, and to avoid being
*forced* to using the *global* keyword in initializer, or being *forced* to
create global variables in the parent process.

The interface is 100% backwards compatible through Python3.x (and perhaps
beyond).
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20180928/03fcc55f/attachment.html>