<br><br><div class="gmail_quote">On Thu, Jul 22, 2010 at 9:55 AM, Samir Faci <span dir="ltr"><<a href="mailto:samir@esamir.com">samir@esamir.com</a>></span> wrote:<br><blockquote class="gmail_quote" style="border-left: 1px solid rgb(204, 204, 204); margin: 0pt 0pt 0pt 0.8ex; padding-left: 1ex;">
I'm sure this is something fairly silly I'm overlooking, but I've been<br>
spending too much time on this... so I figured I should ask the powers<br>
that be.<br><br></blockquote><div><br>This is complicated stuff. Do *not* feel bad. <br></div><div><br>I was working on something similar where I used a non-blocking Queue Server like this one <br><br> <a href="http://www.google.com/codesearch/p?hl=en#EvwmCkrYNSE/src/queue_server.py&l=34&d=0">http://www.google.com/codesearch/p?hl=en#EvwmCkrYNSE/src/queue_server.py&l=34&d=0</a><br>
<br>However, I modified the code so that the worker will periodically post back some results to the server so the log can be displayed. You can run Subprocess in the worker then post stuff back to the Queue. <br><br>The important elements here is how the Queue() waits by default. Also an important to know which *thing* your talking about when your posting anything to the dispatching Queue server. I changed the way the Queue and the id handing works in my own code.<br>
<br>Also, I changed the way stopping stuff works. I wanted to be able to stop on demand and not get a ghost server and make works end gracefully. That was a bit tricky because the code here suggests serve_forever. I put a stop() method on the workers and the server. So in the Workers I sleep() and check _stop periodically. Likewise, when I am looking for Queue objects, put those in loop and set the timeout Queue.get(True,10) so that it does not block. Likewise, you will want to make one final call to the server to release the socket (it may fail, which is fine). If things really get hung on the worker or the server (or if I get impatient) I do a force stop with psutil.<br>
<br>Oh, the other change I made is that I ended up using a WorkerPool Thread that actually has a collection of Workers ready to do *something*. It just starts a bunch of workers that are watching polling waiting for the command to start actually do something from the Server. It has its own stop() which just calls stop on all the worker threads. <br>
<br>Hope this helps,<br><br>-- Brian<br><br><br><br><br> </div></div>