Python concurrent.futures.ProcessPoolExecutor
MRAB
python at mrabarnett.plus.com
Wed Dec 16 12:51:53 EST 2020
On 2020-12-16 16:04, Rob Rosengard wrote:
> Warning: I am new to this group
> Warning: I am not an expert at Python, I've written a few small programs, and spend 20 hours of online classes, and maybe a book or two.
> Warning: I am new to trying to use concurrent.futures.ProcessPoolExecutor
> - Prior to writing this question I updated to Python 3.9 and PyCharm 2020.3. And confirmed the problem still exists.
> - Running on Windows 10 Professional
> - I've been trying to run a simple piece of code to exactly match what I have seen done in various training videos. By I am getting a different and
> unexpected set of results. I.e. the instructor got different results than I did on my computer. My code is very simple:
>
> import concurrent.futures
> import time
>
>
> start = time.perf_counter()
>
>
> def task(myarg):
> print(f'Sleeping one second...{myarg}')
> time.sleep(1)
> return 'Done sleeping...'
>
>
> if __name__ == '__main__':
> with concurrent.futures.ProcessPoolExecutor() as executor:
> future1 = executor.submit(task, 1)
> future2 = executor.submit(task, 2)
> finish = time.perf_counter()
> print(f'Finished in {round(finish-start,2)} seconds')
>
> And the output is:
> Finished in 0.0 seconds
> Finished in 0.0 seconds
> Sleeping one second...1
> Sleeping one second...2
> Finished in 1.14 seconds
>
> Process finished with exit code 0
>
> ---
> QUESTIONS and CONCERNS that I have...
> It seems that both calls to task not only runs that function, but then keeps executing the rest of the main line code. I only expected it to run the function and then immediately quit/disappear. That is, I expect the output to look like (i.e. not having the three lines of "Finished in x.x seconds", rather, just one line like that):
> Sleeping one second...1
> Sleeping one second...2
> Finished in 1.14 seconds
>
> Goal: I need the executor tasks to only run that one function, and then completely go away and stop. Not keep executing more code that doesn't belong to the task function.
>
> I've tried many iterations of this issue, and placed PRINT statements all over to try to track what is going on. And I used if/else statements in the main code, which caused even more problems. I.e. both the IF and the ELSE was executed each time through the code. Which completely blows my mind.
>
> Any thoughts on this? Thanks for your time and help!
>
It imports the module to run the function, so any top-level code _will_
be run.
Here's a modified version:
--8<----------------------------8<--
import concurrent.futures
import time
print('Main code, __name__ is {!a}'.format(__name__))
start = time.perf_counter()
def task(myarg):
print('In task, __name__ is {!a}'.format(__name__))
print(f'Sleeping one second...{myarg}')
time.sleep(1)
return 'Done sleeping...'
if __name__ == '__main__':
with concurrent.futures.ProcessPoolExecutor() as executor:
future1 = executor.submit(task, 1)
future2 = executor.submit(task, 2)
finish = time.perf_counter()
print(f'Finished in {round(finish-start,2)} seconds')
--8<----------------------------8<--
It prints:
--8<----------------------------8<--
Main code, __name__ is '__main__'
Main code, __name__ is '__mp_main__'
Finished in 0.0 seconds
Main code, __name__ is '__mp_main__'
Finished in 0.0 seconds
In task, __name__ is '__mp_main__'
Sleeping one second...1
In task, __name__ is '__mp_main__'
Sleeping one second...2
Finished in 1.8 seconds
--8<----------------------------8<--
Any top-level code that you don't want it to run when it re-imports the
module should be protected by the __name__ test.
More information about the Python-list
mailing list