On 6/12/2020 6:18 AM, Edwin Zimmerman wrote:
On 6/12/2020 5:08 AM, Paul Moore wrote:
On Fri, 12 Jun 2020 at 09:47, Mark Shannon <mark@hotpy.org> wrote:
Starting a new process is cheap. On my machine, starting a new Python process takes under 1ms and uses a few Mbytes. Is that on Windows or Unix? Traditionally, process creation has been costly on Windows, which is why threads, and in-process solutions in general, tend to be more common on that platform. I haven't done experiments recently, but I do tend to avoid multiprocess-type solutions on Windows "just in case". I know that evaluating a new feature based on unsubstantiated assumptions informed by "it used to be like this" is ill-advised, but so is assuming that everything will be OK based on experience on a single platform :-) Here's a test on Windows 10, 4 logical cpus, 8 GB of ram:
timeit.timeit("""multiprocessing.Process(target=exit).start()""",number=100, globals=globals()) 0.6297528999999997 timeit.timeit("""multiprocessing.Process(target=exit).start()""",number=1000, globals=globals()) 40.281721199999964
Or this way:
timeit.timeit("""os.system('python.exe -c "exit()"')""",number=100, globals=globals()) 17.461259299999995
--Edwin For comparison, on a single core linux cloud server with 512 mb of ram:
timeit.timeit("""multiprocessing.Process(target=exit).start()""",number=100, globals=globals()) 0.354354709998006 timeit.timeit("""multiprocessing.Process(target=exit).start()""",number=1000, globals=globals()) 3.847851719998289 So yeah, process creation is still rather costly on Windows.