Multiprocessing vs. concurrent.futures, Linux vs. Windows
Terry Reedy
tjreedy at udel.edu
Mon May 4 16:43:56 EDT 2020
On 5/4/2020 3:26 PM, John Ladasky wrote:
> Several years ago I built an application using multiprocessing. It only needed to work in Linux. I got it working fine. At the time, concurrent.futures did not exist.
>
> My current project is an application which includes a PyQt5 GUI, and a live video feed with some real-time image processing. Running all this in one process resulted in unacceptable performance, so I'm trying to move all of the heavy video work into its own process. I only need to define one child process. I don't need a Pool. The child Process can run indefinitely, and it will communicate multiple messages to the main process using a Pipe.
>
> I built a working example in Linux, but it hangs in Windows. I built a minimum example. My problem has nothing to do with PyQt. In Windows, my example hangs when I try to create the child Process. Code:
>
>
> import os
> from multiprocessing import Pipe, Process
>
> def child_app():
> inlet.send("Child process id = {}".format(os.getpid()))
>
> if __name__ == "__main__":
> outlet, inlet = Pipe()
> print("Parent process id =", os.getpid())
> child_process = Process(target=child_app) # Windows hangs here
> child_process.start()
> print(outlet.recv())
> child_process.join()
> child_process.close()
> print("Program complete.")
>
>
> I'm working in Python 3.7.6 on Windows 10, and 3.7.5 on Ubuntu Linux 19.10.
Does the minimal example in the doc work for you?
(IE, do you only have a problem with Pipe?)
from multiprocessing import Process
def f(name):
print('hello', name)
if __name__ == '__main__':
p = Process(target=f, args=('bob',))
p.start()
p.join()
How about the Pipe example?
from multiprocessing import Process, Pipe
def f(conn):
conn.send([42, None, 'hello'])
conn.close()
if __name__ == '__main__':
parent_conn, child_conn = Pipe()
p = Process(target=f, args=(child_conn,))
p.start()
print(parent_conn.recv()) # prints "[42, None, 'hello']"
p.join()
If this does not work on Windows, the example or doc should be changed.
But I believe I tested it once. Note that unlike your code, the
child_conn is sent as argument. The relation between the module code
and child processes is different on Windows than *nix.
More information about the Python-list
mailing list