Speed up with threads

Syver Enstad syver-en+usenet at online.no
Sun Aug 4 14:23:50 CEST 2002

Rhymes <raims at dot.com> writes:

> On Sat, 03 Aug 2002 11:16:36 +0200, Thomas Jensen <spam at ob_scure.dk>
> wrote:
> >Have a look at the Queue module.
> I read both the Queue doc and the Aahz slides but I still have
> problems... what goes in the Queue? How to use get() and put(), I have
> to scan ports not to share data... I think

I think what is meant is that instead of you creating a new Scanner
object for each port scan you should implement your run method in the
Scanner like this:

pseudo code:

def run(self):
    while 1:
        host, port = self._queue.get()
        ..scan code here..

in the main class instead of creating new threads, just allocate a
fixed number of Scanner objects and keep them in a list, then:

def scan(self, host, start, stop):
        self.port = start
        while self.port < stop:
            self.port = self.port + 1
            self._queue.put((host, self.port))

You'll have to make the two classes share the same Queue, maybe like

In the ctor for the main class, or an init method of some kind:
self._queue = Queue.Queue()

in a loop creating the Scanners:

Experiment with the number of scanners to find out what gives you the
best performance.

BTW, this technique of using threads is called thread pooling, because
you operate with a pool of threads instead of creating new ones all
the time.


Vennlig hilsen 

Syver Enstad

More information about the Python-list mailing list