Processes not exiting

[ma3mju]

On 31 July, 11:27, Piet van Oostrum <p... at cs.uu.nl> wrote:
> >>>>> ma3mju <matt.u... at googlemail.com> (m) wrote:
> >m> Hi all,
> >m> I'm having trouble with multiprocessing I'm using it to speed up some
> >m> simulations, I find for large queues when the process reaches the
> >m> poison pill it does not exit whereas for smaller queues it works
> >m> without any problems. Has anyone else had this trouble? Can anyone
> >m> tell me a way around it? The code is in two files below.
>
> How do you know it doesn't exit. You haven't shown any of your output.
>
> I did discover a problem in your code, but it should cause an exception:
>
> >m> #set off some of the easy workers on the hard work (maybe double
> >m> number of hard)
> >m> for i in range(0,num_hard_workers):
> >m>     hard_work_queue.put(None)
> >m>     hard_workers.append(multiprocessing.Process
> >m> (target=GP.RandomWalkGeneralizationErrorParallel,args=
> >m> (hard_work_queue,result_queue,)))
> >m> #wait for all hard workers to finish
> >m> for worker in hard_workers:
> >m>     worker.join()
>
> Here you create new hard workers, but you never start them. The join
> should then give an exception when it reaches these.
> --
> Piet van Oostrum <p... at cs.uu.nl>
> URL:http://pietvanoostrum.com[PGP 8DAE142BE17999C4]
> Private email: p... at vanoostrum.org

Ok thanks I'll change that In a sec, It never reaches that bit of code
because the easy_workers don't exit so it never gets past the join().

As far as running it goes I get the datapt and number of points
printed to the list for everything in both queues. When it reaches the
end of either queue I get "Poison" on the screen then "here" for each
process but I don't get "worker joined" and as expected don't get
"this should not appear". If I have a look at the processes running
after all queues are supposed to have finished I see all of them
running taking little or no resources. This is running on Ubuntu
Jaunty at home and the same happens on the Debian machine at uni.

The weird thing is that if I run them with less points the processes
do manage to exit.

Thanks

Matt