[Tutor] Multiprocessing
Cameron Simpson
cs at cskk.id.au
Tue Sep 22 21:08:44 EDT 2020
On 22Sep2020 08:03, Stephen M Smith <stephen.m.smith at comcast.net> wrote:
>OK but I why does it run 17 times.? There were 10 copies fired up in the
>version you saw (I think) so I sort of get why it would run once originally
>and 1 more time for each of the multiprocessing jobs, but that only gets me
>to 11. 17 is not a number I can resolve.
I've just run your code here. I think the ProcessPoolExecutor prefills
16 subprocesses hoping that by the time you call executor.submit there
is a subprocess instantly ready to go (as opposed to, when you ask,
making a whole new Python process, importing your module, _then_ calling
the function). This is in anticipation of running many subprocesses,
not a single immediate batch like your test code.
So what you see is your top level "starting up" message, then 16 (one
for each premade subprocess). Then you only use 10 of them.
So the pool is preparing subprocesses as soon as you make the pool,
_not_ on demand as you ask things to happen.
Cheers,
Cameron Simpson <cs at cskk.id.au>
More information about the Tutor
mailing list