how to start thread by group?

[bieffe62 at gmail.com]

On 7 Ott, 06:37, "Gabriel Genellina" <gagsl-... at yahoo.com.ar> wrote:
> En Mon, 06 Oct 2008 11:24:51 -0300, <bieff... at gmail.com> escribió:
>
> > On 6 Ott, 15:24, oyster <lepto.pyt... at gmail.com> wrote:
> >> my code is not right, can sb give me a hand? thanx
>
> >> for example, I have 1000 urls to be downloaded, but only 5 thread at  
> >> one time
> > I would restructure my code with someting like this ( WARNING: the
> > following code is
> > ABSOLUTELY UNTESTED and shall be considered only as pseudo-code to
> > express my idea of
> > the algorithm (which, also, could be wrong:-) ):
>
> Your code creates one thread per url (but never more than MAX_THREADS  
> alive at the same time). Usually it's more efficient to create all the  
> MAX_THREADS at once, and continuously feed them with tasks to be done. A  
> Queue object is the way to synchronize them; from the documentation:
>
> <code>
>  from Queue import Queue
>  from threading import Thread
>
> num_worker_threads = 3
> list_of_urls = ["http://foo.com", "http://bar.com",
>                  "http://baz.com", "http://spam.com",
>                  "http://egg.com",
>                 ]
>
> def do_work(url):
>      from time import sleep
>      from random import randrange
>      from threading import currentThread
>      print "%s downloading %s" % (currentThread().getName(), url)
>      sleep(randrange(5))
>      print "%s done" % currentThread().getName()
>
> # from this point on, copied almost verbatim from the Queue example
> # at the end ofhttp://docs.python.org/library/queue.html
>
> def worker():
>      while True:
>          item = q.get()
>          do_work(item)
>          q.task_done()
>
> q = Queue()
> for i in range(num_worker_threads):
>       t = Thread(target=worker)
>       t.setDaemon(True)
>       t.start()
>
> for item in list_of_urls:
>      q.put(item)
>
> q.join()       # block until all tasks are done
> print "Finished"
> </code>
>
> --
> Gabriel Genellina


Agreed.
I was trying to do what the OP was trying to do, but in a way that
works.
But keeping the thread alive and feeding them the URL is a better
design, definitly.
And no, I don't think its 'premature optimization': it is just
cleaner.

Ciao
------
FB