My Object pool isnt working

[sam]

Hi,

I need to pool some database connections so i thought I'd write a
object pool. I thought I'd use a design I've used in Java before -
having a locked and unlocked hashtable and keeping references of the
connections in two states (i.e. allocated and unallocated). The
problem is that the total number of connections always seems to rise.
Also the total number of connections is less than the sum of the
lengths of the hashtables (dictionaries) which shouldn't happen. I
believe the problems are connected. As I said I've used this design
before in Java and its been fine. The only difference I can think of
(barring shoddy implementation, which I can't rule out) is that
Hashtables in Java are thread safe, but I'm not sure if the built in
dict type in python is.

I've decided to use jonpy's dbpooling module
(http://sourceforge.net/projects/jonpy) instead of writing my own now,
but I'd still like to know whats going on. I have tested on python
1.52 and python 2.2 (both on win2k) and get the same result. All
thoughts appriciated

Regards

Sam Owen

<code>
from Threading import *
import time
import random
import sys

class Object:
    def sayHello(self, caller):
        print "hello", self, caller 
        time.sleep(int(random.random() * 5))

class ObjectPool:
    LOCKED= {}
    UNLOCKED= {}
    sema_connect = Semaphore(1)
    sema_return = Semaphore(1)
    expiretime = 60 #30 mins
    totalConns=0
    numAllocated = 0
    numReturned = 0

    def getConnection(self):
        try:
            self.sema_connect.acquire()
            if len(self.UNLOCKED):
                for ts in self.UNLOCKED.keys() :
                    if ts + self.expiretime < time.time():
                        print "deleting ..."
                        self.totalConns = self.totalConns-1
                        del self.UNLOCKED[ts]
                    else:
                        print "allocating ..."
                        self.numAllocated = self.numAllocated+1
                        o = self.UNLOCKED[ts]
                        self.LOCKED[o]=1
                        del self.UNLOCKED[ts]
                        return o
            print "creating ..."
            self.numAllocated = self.numAllocated+1
            self.totalConns = self.totalConns+1
            o = Object()
            self.LOCKED[o] = 1
            return o
        finally:
            self.sema_connect.release()    

    def returnConnection(self, con):
        try:
            self.sema_return.acquire()
            if self.LOCKED.has_key(con):
                print "returning"
                self.numReturned = self.numReturned +1
                self.UNLOCKED[time.time()]=con
                del self.LOCKED[con]
                return
            else:
                print con, "was not found in LOCKED ?????"
        finally:
            self.sema_return.release()

class ObjectUser(Thread):
    def __init__(self, pool):
        self.pool = pool
        Thread.__init__(self)
    def run(self):
        while 1:
            o = pool.getConnection()
            o.sayHello(self)
            pool.returnConnection(o)
            time.sleep(int(random.random() * 5))
        
if __name__ == '__main__':
    pool = ObjectPool()
    for i in range(2):
        thread = ObjectUser(pool)
        thread.start()
    while 1:
        print "LOCKED --> ", len(pool.LOCKED), "\nUNLOCKED -->",
len(pool.UNLOCKED), "\ntotal conns", pool.totalConns
        print "allocates", pool.numAllocated, "\nreturns",
pool.numReturned
        time.sleep(5)
</code>

<sample output>
creating ...
LOCKED -->  1
UNLOCKED --> 0
total conns 1
allocates 1
returns 0
creating ...
hello <__main__.Object instance at 8797a8> <ObjectUser(Thread-2,
started)>
hello <__main__.Object instance at 879790> <ObjectUser(Thread-1,
started)>
returning
allocating ...
hello <__main__.Object instance at 879790> <ObjectUser(Thread-1,
started)>
returning
LOCKED -->  1
UNLOCKED --> 1
total conns 2
allocates 3
returns 2
allocating ...
hello <__main__.Object instance at 8797a8> <ObjectUser(Thread-2,
started)>
returning
returning
allocating ...
hello <__main__.Object instance at 8797a8> <ObjectUser(Thread-2,
started)>
returning
allocating ...
hello <__main__.Object instance at 8797a8> <ObjectUser(Thread-2,
started)>
creating ...
hello <__main__.Object instance at 85efe8> <ObjectUser(Thread-1,
started)>
LOCKED -->  2
UNLOCKED --> 0
total conns 3
allocates 7
returns 5
returning
returning
allocating ...
hello <__main__.Object instance at 8797a8> <ObjectUser(Thread-1,
started)>
returning
</sample output>