multithreading

[Skip Montanaro]

    >> The number of active threads is effectively capped by a Queue object
    >> which contains a set of cached MySQLdb connection objects.

    Peter> Cool!  Do you mean you prepare the Queue ahead of time with a
    Peter> fixed number of those objects, then any attempt to create a new
    Peter> thread blocks on the Queue until a previous thread has terminated
    Peter> and released its resource back into the Queue with a put(),
    Peter> presumably inside a 'finally' block?

More or less, yes.  Once a thread is finished with a db connection, it
places it back on the queue.  It is not obligated to terminate at that
point.  It can try and grab other resources it needs.

Every chunk of code where I lock something looks something like:

    self.cache_lock.acquire()
    try:
        fiddle_the_cache...
    finally:
        self.cache_lock.release()

The dance with the Queue object full of database connections doesn't
actually use try/finally because it has a fairly weird set of interactions -
restarting some queries if they fail for specific reasons, reconnecting if a
connection has logged too many errors, etc.  I'm fairly careful to catch all
the possible exceptions (yes, I have a general except: clause).

-- 
Skip Montanaro (skip at pobox.com - http://www.mojam.com/)
"Excellant Written and Communications Skills required" - seen on chi.jobs