collecting results in threading app

[George Sakkis]

On Apr 4, 10:42 am, Gerardo Herzig <gher... at fmed.uba.ar> wrote:

> Hi all. Newbee at threads over here. Im missing some point here, but cant
> figure out which one.
>
> This little peace of code executes a 'select count(*)' over every table
> in a database, one thread per table:
> <code>
> class TableCounter(threading.Thread):
>     def __init__(self, conn, table):
>         self.connection = connection.Connection(host=conn.host,
> port=conn.port, user=conn.user, password='', base=conn.base)
>         threading.Thread.__init__(self)
>         self.table = table
>
>     def run(self):
>         result =  self.connection.doQuery("select count(*) from %s" %
> self.table, [])[0][0]
>         print result
>         return result
>
> class DataChecker(metadata.Database):
>
>     def countAll(self):
>         for table in self.tables:
>             t = TableCounter(self.connection, table.name)
>             t.start()
>         return
> </code>
>
> It works fine, in the sense that every run() method prints the correct
> value.
> But...I would like to store the result of t.start() in, say, a list. The
> thing is, t.start() returns None, so...what im i missing here?
> Its the desing wrong?

The simplest way is to just store it as an attribute in the
TableCounter instance:

     def run(self):
         self.result =  self.connection.doQuery(...)

Another alternative is to add it to a Queue. You can't use a list
unless you protect with a lock to prevent concurrent append()s, but
that's what Queues do anyway [1].

Regardless of where the results are stored, a second issue which you
don't address here is, how do you know that a given result or all
results are done ? Again there are several alternatives, but Python
2.5 adds two convenient Queue methods for this, task_done() and
join(). Check out the example at the bottom of the Queue doc page [2]
to see how it works.

HTH,
George


[1] http://docs.python.org/lib/module-Queue.html
[2] http://docs.python.org/lib/QueueObjects.html