[medusa] Strange exception in ZServer/medusa/asynchat.py

[Skip Montanaro]

I'm using Medusa by way of Zope's ZServer. I have ZServer sitting behind an
Apache proxy and am bombing it from a threaded client script that reads a
web server log file as input and throws requests at the Apache server. I
can vary the number of threads to adjust the load. I normally run it with
five simultaneous threads.

 by default, the system runs briefly then (as they say), "Whammo! Blammo!", I
get the following error:

Unhandled exception in thread:
Traceback (innermost last):
File "/home/dolphin/skip/src/Zope/ZServer/PubCore/ZServerPublisher.py", line 97, in __init__
response._finish()
File "/home/dolphin/skip/src/Zope/ZServer/HTTPResponse.py", line 209, in _finish
self.stdout.close()
File "/home/dolphin/skip/src/Zope/ZServer/HTTPResponse.py", line 235, in close
self._channel.push(CallbackProducer(self._channel.done))
File "/home/dolphin/skip/src/Zope/ZServer/HTTPServer.py", line 307, in push
if send: self.initiate_send()
File "/home/dolphin/skip/src/Zope/ZServer/medusa/asynchat.py", line 200, in initiate_send
self.refill_buffer()
File "/home/dolphin/skip/src/Zope/ZServer/medusa/asynchat.py", line 184, in refill_buffer
self.producer_fifo.pop()
File "/home/dolphin/skip/src/Zope/ZServer/medusa/asynchat.py", line 255, in pop
del self.list[0]
IndexError: list assignment index out of range

If you look at the code in asynchat.fifo.pop, the error can only be
explained by the fact that two threads are modifying the fifo
simultaneously, since access to self.list[0] succeeded in the statement
before the statement that throws the IndexError:

def pop (self):
if self.list:
result = (1, self.list[0])	# non-null list here
del self.list[0]	# empty list here
else:
result = (0, None)
return result

I modified the fifo class to lock access to its list:

class fifo:
def __init__ (self, list=None):
self.lock = thread.allocate_lock()
self.lock.acquire()
if not list:
self.list = []
else:
self.list = list
self.lock.release()

def __len__ (self):
self.lock.acquire()
l = len(self.list)
self.lock.release()
return l

def first (self):
self.lock.acquire()
item = self.list[0]
self.lock.release()
return item

def push (self, data):
self.lock.acquire()
self.list.append (data)
self.lock.release()

def pop (self):
self.lock.acquire()
if self.list:
result = (1, self.list[0])
del self.list[0]
else:
result = (0, None)
self.lock.release()
return result

but this didn't help:

Unhandled exception in thread:
Traceback (innermost last):
File "/home/dolphin/skip/src/Zope/ZServer/PubCore/ZServerPublisher.py", line 97, in __init__
response._finish()
File "/home/dolphin/skip/src/Zope/ZServer/HTTPResponse.py", line 209, in _finish
self.stdout.close()
File "/home/dolphin/skip/src/Zope/ZServer/HTTPResponse.py", line 235, in close
self._channel.push(CallbackProducer(self._channel.done))
File "/home/dolphin/skip/src/Zope/ZServer/HTTPServer.py", line 307, in push
if send: self.initiate_send()
File "/home/dolphin/skip/src/Zope/ZServer/medusa/asynchat.py", line 200, in initiate_send
self.refill_buffer()
File "/home/dolphin/skip/src/Zope/ZServer/medusa/asynchat.py", line 175, in refill_buffer
p = self.producer_fifo.first()
File "/home/dolphin/skip/src/Zope/ZServer/medusa/asynchat.py", line 254, in first
item = self.list[0]
IndexError: list index out of range

This error also looks like a threading problem. When refill_buffer tests
self.producer.fifo, it sees something, but by the time it calls fifo.first,
the list has evaporated:

while 1:
if len(self.producer_fifo):	# non-null list here
p = self.producer_fifo.first()	# null list here

Any ideas? Is there a way to easily turn off threading? It seems that the
data structures are not properly protected.


Skip Montanaro	| Mojam: "Uniting the World of Music" http://www.mojam.com/
skip@m... | Musi-Cal: http://www.musi-cal.com/
518-372-5583

------------------------------------------------------------------------

eGroups.com home: http://www.egroups.com/group/medusa
http://www.egroups.com - Simplifying group communications