Thread Question

[Ritesh Raj Sarraf]

Duncan,

I couldn't make out much from the code.
Instead this is what I did.

threads = []
        nloops = range(len(lRawData))
        for i in nloops:
            (sUrl, sFile, download_size, checksum) =
stripper(lRawData[i])
            t = threading.Thread(target=download_from_web, args=(sUrl,
sFile, sSourceDir, None))
            # = pypt_thread(download_from_web, i,
stripper(lRawData[i]))
            threads.append(t)

        i = 0
        join_i = 0
        while i < nloops:
            counter = 0
            while counter < 3:
                threads[i].start()
                counter += 1
                i += 1
            counter = 0
            join_i = i - 3
            while counter < 3:
                threads[join_i].join()
                counter += 1
                join_i += 1

Is this correct ? Comments!!

Ritesh


Duncan Booth wrote:
> Ritesh Raj Sarraf wrote:
>
> > I'm planning to implement threads in my application so that multiple
> > items can be downloaded concurrently. I want the thread option to be
> > user-defined.
> >
> > Looking at the documentation of threads (Core Python Programming), I've
> > noticed that all threads are executed a once. Depending upon what they
> > are doing, some finish early and some later.
> >
> > But I want to implement something like:
> >
> > for item in list_items:
> >     for num in thread_args:
> >        thread[num].start()
> >        thread[num].start()
> >
> > Is this the correct way of threading applications ?
> > This is the first time I'd be doing threading. So was looking for
> > comments and suggestions.
> >
>
> What you want is to use a pool of threads so that you can configure how
> many requests are issued at a time (you don't want to try to issue 100
> requests all in parallel). You can communicate with the threads through a
> Queue.
>
> So if the code for a thread looks like:
>
>    def run(request, response):
>        while 1:
>            item = request.get()
>            if item is None:
>                 break
>            response.put(download_from_web(item))
>
> # your main loop can be something like:
>
> requestQueue = Queue()
> responseQueue = Queue()
> thread_pool = [
>         Thread(target=run, args=(requestQueue, responseQueue)
>         for i in range(numthreads)]
> for t in thread_pool: t.start()
>
> for item in list_items:
>      requestQueue.put(item)
>
> for i in range(len(list_items)):
>     response = responseQueue.get()
>     handle_response(response)
>
> # and then to shut down the threads when you've finished:
> for t in thread_pool:
>     requestQueue.put(None)
> for t in thread_pool:
>      t.join()