Processes not exiting

[MRAB]

ma3mju wrote:
> Hi all,
> 
> I'm having trouble with multiprocessing I'm using it to speed up some
> simulations, I find for large queues when the process reaches the
> poison pill it does not exit whereas for smaller queues it works
> without any problems. Has anyone else had this trouble? Can anyone
> tell me a way around it? The code is in two files below.
> 
[snip]
> 
> #get number of cores and set the number on concurrent processes
> num_hard_workers = 2
> num_workers = multiprocessing.cpu_count()*1.5
> easy_workers = []
> hard_workers = []
> #add poison pill for each worker and create the worker
> for i in range(0,num_workers-num_hard_workers):
>     easy_work_queue.put(None)
>     easy_workers.append(multiprocessing.Process
> (target=GP.RandomWalkGeneralizationErrorParallel,args=
> (easy_work_queue,result_queue,)))
> for i in range(0,num_hard_workers):
>     hard_work_queue.put(None)
>     hard_workers.append(multiprocessing.Process
> (target=GP.RandomWalkGeneralizationErrorParallel,args=
> (hard_work_queue,result_queue,)))
> 
You have 2 hard workers and ceil(CPU_count * 1.5) - 2 easy workers.
What if the number of CPUs was 1? That would give 2 hard and 0 easy!

Also, I recommend that you put only 1 'poison pill' in each queue and
have the workers put it back when they see it.