Multiprocessing vs. concurrent.futures, Linux vs. Windows

Terry Reedy tjreedy at udel.edu
Mon May 4 16:43:56 EDT 2020


On 5/4/2020 3:26 PM, John Ladasky wrote:
> Several years ago I built an application using multiprocessing.  It only needed to work in Linux.  I got it working fine.  At the time, concurrent.futures did not exist.
> 
> My current project is an application which includes a PyQt5 GUI, and a live video feed with some real-time image processing.  Running all this in one process resulted in unacceptable performance, so I'm trying to move all of the heavy video work into its own process.  I only need to define one child process.  I don't need a Pool.  The child Process can run indefinitely, and it will communicate multiple messages to the main process using a Pipe.
> 
> I built a working example in Linux, but it hangs in Windows.  I built a minimum example.  My problem has nothing to do with PyQt.  In Windows, my example hangs when I try to create the child Process.  Code:
> 
> 
> import os
> from multiprocessing import Pipe, Process
> 
> def child_app():
>      inlet.send("Child process id = {}".format(os.getpid()))
> 
> if __name__ == "__main__":
>      outlet, inlet = Pipe()
>      print("Parent process id =", os.getpid())
>      child_process = Process(target=child_app)  # Windows hangs here
>      child_process.start()
>      print(outlet.recv())
>      child_process.join()
>      child_process.close()
>      print("Program complete.")
> 
> 
> I'm working in Python 3.7.6 on Windows 10, and 3.7.5 on Ubuntu Linux 19.10.

Does the minimal example in the doc work for you?
(IE, do you only have a problem with Pipe?)

from multiprocessing import Process

def f(name):
     print('hello', name)

if __name__ == '__main__':
     p = Process(target=f, args=('bob',))
     p.start()
     p.join()

How about the Pipe example?

from multiprocessing import Process, Pipe

def f(conn):
     conn.send([42, None, 'hello'])
     conn.close()

if __name__ == '__main__':
     parent_conn, child_conn = Pipe()
     p = Process(target=f, args=(child_conn,))
     p.start()
     print(parent_conn.recv())   # prints "[42, None, 'hello']"
     p.join()

If this does not work on Windows, the example or doc should be changed.
But I believe I tested it once. Note that unlike your code, the 
child_conn is sent as argument.  The relation between the module code 
and child processes is different on Windows than *nix.



More information about the Python-list mailing list