A question concerning named pipes, multiprocessing.Queue (multiprocessing.queues.JoinableQueue)
I raised this issue and question on StackExchange and #python (FreeNode) and have received little or no feedback. I fear that the only answer will lie in profiling the python interpreter itself, which is beyond the scope of my capabilities at present. The original question can be found here: http://stackoverflow.com/questions/38637282/multiprocessing-queue-seems-to-g... Cliffs: after about one to two hours of processing, the consumer process reading the queue terminates on timeout during queue.get(). The thread writing objects to the queue receives no exception continues writing to the queue. The thread keeps track of each child process using a simple local list of processes, and each time an object is added to the queue, the processes objects (consumers) are checked with .is_alive(). When the child processes suicide on timeout, is_alive() continues to return True, so they are not garbage collected nor are new processes started. I sincerely apologize if my own understanding of the documentation is at fault, but to the best of my knowledge this code should work according to the documentation.
participants (1)
-
Matt Gregory