New submission from Christian Ullrich <chris@chrullrich.net>: Using concurrent.futures.ProcessPoolExecutor on Windows fails immediately with a lot of exceptions of the "access denied", "file not found", and "invalid handle" varieties. Running the script that creates the ProcessPoolExecutor from the main system-wide installation works correctly. Due to Windows' infamous lack of fork(), ProcessPoolExecutor launches its worker processes by setting up an inheritable handle to a pipe and passing the handle on the command line. In a venv situation, it appears that the venv's python.exe internally launches the parent environment's python.exe and passes on its command line, but not its handle table. This sub-subprocess therefore does not have the original handle, and may have a different handle at the same index. Output of the ProcessPoolExecutor example program from the docs when run with the main installation: C:\Daten>py cft.py 112272535095293 is prime: True 112582705942171 is prime: True 112272535095293 is prime: True 115280095190773 is prime: True 115797848077099 is prime: True 1099726899285419 is prime: False Output when run using a venv: C:\Daten>pyv\v37\Scripts\python.exe cft.py Process SpawnProcess-4: Traceback (most recent call last): File "C:\Program Files\Python37\lib\multiprocessing\process.py", line 297, in _bootstrap self.run() File "C:\Program Files\Python37\lib\multiprocessing\process.py", line 99, in run self._target(*self._args, **self._kwargs) File "C:\Program Files\Python37\lib\concurrent\futures\process.py", line 226, in _process_worker call_item = call_queue.get(block=True) File "C:\Program Files\Python37\lib\multiprocessing\queues.py", line 93, in get with self._rlock: File "C:\Program Files\Python37\lib\multiprocessing\synchronize.py", line 95, in __enter__ return self._semlock.__enter__() PermissionError: [WinError 5] Access is denied Process SpawnProcess-5: Traceback (most recent call last): File "C:\Program Files\Python37\lib\multiprocessing\process.py", line 297, in _bootstrap self.run() File "C:\Program Files\Python37\lib\multiprocessing\process.py", line 99, in run self._target(*self._args, **self._kwargs) File "C:\Program Files\Python37\lib\concurrent\futures\process.py", line 226, in _process_worker call_item = call_queue.get(block=True) File "C:\Program Files\Python37\lib\multiprocessing\queues.py", line 93, in get with self._rlock: File "C:\Program Files\Python37\lib\multiprocessing\synchronize.py", line 95, in __enter__ return self._semlock.__enter__() PermissionError: [WinError 5] Access is denied Traceback (most recent call last): File "cft.py", line 28, in <module> main() File "cft.py", line 24, in main for number, prime in zip(PRIMES, executor.map(is_prime, PRIMES)): File "C:\Program Files\Python37\lib\concurrent\futures\process.py", line 476, in _chain_from_iterable_of_lists for element in iterable: File "C:\Program Files\Python37\lib\concurrent\futures\_base.py", line 586, in result_iterator yield fs.pop().result() File "C:\Program Files\Python37\lib\concurrent\futures\_base.py", line 432, in result return self.__get_result() File "C:\Program Files\Python37\lib\concurrent\futures\_base.py", line 384, in __get_result raise self._exception concurrent.futures.process.BrokenProcessPool: A process in the process pool was terminated abruptly while the future was running or pending. ---------- assignee: docs@python components: Documentation, Library (Lib), Windows messages: 334142 nosy: chrullrich, docs@python, paul.moore, steve.dower, tim.golden, zach.ware priority: normal severity: normal status: open title: concurrent.futures.ProcessPoolExecutor does not work in venv on Windows type: behavior versions: Python 3.7 _______________________________________ Python tracker <report@bugs.python.org> <https://bugs.python.org/issue35797> _______________________________________