my multi-download program can't finish

[MRAB]

On 08/09/2011 05:24, 守株待兔 wrote:
> here is the program,
>
> # basic structure,omit something
> import Queue
> import httplib2
> import threading
> jobs = Queue.Queue()
> name=something   #omit ,it is a web list to download
> for  x  in  name:
>   jobs.put(x)
>
> def download():
>     while not jobs.empty():
>             try:
>                 url = jobs.get()
>                 hx = httplib2.Http()
>                 resp, content = hx.request(url, headers=headers)
>                 jobs.task_done()
>             except:
>                 print  "wrong" , url
>
> if __name__ == '__main__':
>
>     for i in range(10):
>           threading.Thread(target=download).start()
>     jobs.join()
>
> when it run ,it can download someting ,
> it is strang:there is wrong output ,some web can't get,but  the 
> program can't stop,it stay ,run ,can't fininsh,
> i don't know why?
>
The line:

jobs.join()

will wait until every job has been marked as done by:

jobs.task_done()

In function "download", if there's an exception, it will go to the 
exception handler and print a message, but there's no:

jobs.task_done()

there to tell the queue that the job has been done.

You need to tell it when a job has been processed. It doesn't care 
whether a job succeeded or failed, only whether it has been processed.