Too many threads

mark.pelletier at asrcms.com mark.pelletier at asrcms.com
Thu Sep 16 18:55:51 CEST 2010


For some reason, the tasks I put into my thread pool occasionally get
run more than once.

Here's the code:

#
-------------------------------------------------------------------------------------------------------------------
from threading import Thread
from queue import Queue
import subprocess


class ThreadPool(object):
    def __init__(self, thread_count):
        '''
        Argument thread_count is the maximum number of theads.
        '''
        self.thread_count = thread_count
        self.queue = Queue()
        # create and start the threads
        for i in range(self.thread_count):
            t = Thread(target=self._worker)
            t.daemon = True
            t.start()

    def _worker(self):
        '''
        A "private" method that pulls tasks off the queue and does
something
        with them.
        '''
        while True:
            item = self.queue.get()
            print(item)
            self.queue.task_done()

    def run(self, tasklist):
        '''
        Put tasks in the queue.
        '''
        for item in tasklist:
            self.queue.put(item)
        # block until all threads are done.
        self.queue.join()
# - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
-
tasklist = [ 'task1', 'task2', 'task3', 'task4', 'task5', 'task6',
'task7',
'task8', 'task9', 'task10', 'task11', 'task12', 'task13', 'task14',
'task15',
'task16']

if __name__ == '__main__':

    t = ThreadPool(3)
    t.run(tasklist)

#---------------------------------------------------------------------------------------------------

And here's some typical output:

task1
task2
task2
task3
task4
task5
task5
task6
task7
task8
task8
task9
task8
task9
task10
task11
task11
task12
task13
task13
task14
task15
task15
task16


I only want a task to get fired off once.  What am I doing wrong?

Thanks!



More information about the Python-list mailing list