Queue.get_nowait() sometimes bogusly returns Empty

[Geoffrey Talvola]

Geoffrey Talvola wrote:
> In case you're wondering, this is in Webware's WebKit 
> application server
> where there is a pool of N threads, and a queue of 
> (non-thread-safe) servlet
> instances acting as a cache.  We only want to create servlet 
> instances when
> all existing instances are in-use.  When a given thread needs 
> a servlet, it
> basically does:
> 
> 	try:
> 		instance = cache.get_nowait()
> 	except Queue.Empty:
> 		instance = new_instance()
> 	# use the instance...
> 	cache.put(instance)
> 
> The problem is that every once in a while get_nowait() 
> returns Empty when
> the queue isn't actually empty, so the size of the cache 
> grows slowly and
> unboundedly over time.  But I'd prefer for the cache to 
> contain at most N
> instances where N is the number of threads.
> 
> - Geoff

Now that I think of it, maybe a Queue just isn't the right tool for this
job.  Queues are basically designed for the case where you want block on an
empty queue.  But for this use I never want blocking.  Perhaps a simple
combination of a list and a lock would be better, or even a list alone
without a lock:

	try:
		instance = cachelist.pop()
	except IndexError:
		instance = new_instance()
	# use the instance
	cachelist.append(instance)

As long as pop() and append() are atomic, which I believe they are, then
this ought to work.

(This actually implements a stack, not a queue, but for this application it
doesn't matter.)

- Geoff