Threaded for loop

skip at pobox.com skip at pobox.com
Sat Jan 13 21:53:29 CET 2007


    John> I want to do something like this:

    John> for i = 1 in range(0,N):
    John>  for j = 1 in range(0,N):
    John>    D[i][j] = calculate(i,j)

    John> I would like to now do this using a fixed number of threads, say
    John> 10 threads.  What is the easiest way to do the "parfor" in python?

I'd create a queue containing 10 tokens.  Pull a token off the queue, invoke
the thread with the parameters for its chunk, have it compute its bit, lock
D, update it, unlock it, then return the token to the token queue. Sketching
(and completely untested):

    # Calculate one row of D
    def calcrow(i, N, token, Tqueue, Dqueue):
        d = [0.0] * N
        for j in range(N):
            d[j] = calculate(i, j)
        D = Dqueue.get()
        D[i][:] = d
        Dqueue.put(D)
        Tqueue.put(token)

    # This queue limits the number of simultaneous threads
    Tqueue = Queue.Queue()
    for i in range(10):
        Tqueue.put(i)

    # This queue guards the shared matrix, D
    Dqueue = Queue.Queue()
    D = []
    for i in range(N):
        D.append([0.0] * N)
    Dqueue.put(D)

    for i in range(N):
        token = Tqueue.get()
        t = threading.Thread(target=calcrow, args=(i, N, token, Tqueue,
                                                   Dqueue))
        t.start()

Skip



More information about the Python-list mailing list