[Python-Dev] bpo-34837: Multiprocessing.Pool API Extension - Pass Data to Workers w/o Globals
seanharr11 at gmail.com
Fri Sep 28 17:07:33 EDT 2018
I am proposing an extension to the multiprocessing.Pool API that allows for
an alternative way to pass data to Pool worker processes, *without* using
A PR has been opened <https://github.com/python/cpython/pull/9627>,
extensive test coverage is also included, with all tests & CI passing on
Please see this blog post
for details, motivation, and use cases of the API extension before reading
In *short*, the implementation of the feature works as follows:
1. Exposes a kwarg on Pool.__init__ called `expect_initret`, that
defaults to False. When set to True:
1. Capture the return value of the initializer kwarg of Pool
2. Pass this value to the function being applied, as a kwarg.
Again, in *short,* the motivation of the feature is to provide an explicit
"flow of data" from parent process to worker process, and to avoid being
*forced* to using the *global* keyword in initializer, or being *forced* to
create global variables in the parent process.
The interface is 100% backwards compatible through Python3.x (and perhaps
-------------- next part --------------
An HTML attachment was scrubbed...
More information about the Python-Dev