Memory leak with twisted web server on QNX
Hi, I use twisted as a very simple web server to serve python scripts. I now have a problem of memory leak on the server. I use Python 2.2.1 on QNX 6.3, Twisted version is 1.3.0. I made the following test scripts : Web server from twisted.web import server, resource,static,script from twisted.internet import reactor root = static.File("site") root.processors = {'.py': script.PythonScript, '.rpy': script.ResourceScript} site = server.Site(root) reactor.listenTCP(8500, site) reactor.run() Test script (.rpy file) from twisted.web import resource class Test(resource.Resource): isLeaf = True def render_GET(self, request): return """<html> <head> <meta http-equiv="refresh" content="5"> </head> Hello, world!</html>""" resource = Test() About every 4 "refresh", the memory used by my server increases by 96k, with nor release of memory. I checked the object number by watching len(gc.get_objects()), but it is stable. Any idea to help me to solve this problem, my server is expected to run continuously. Thanks. Denis
On Wed, Oct 19, 2005 at 10:15:24AM +0200, denis.leborgne@rieter.com wrote:
Hi,
I use twisted as a very simple web server to serve python scripts. I now have a problem of memory leak on the server. I use Python 2.2.1 on QNX 6.3, Twisted version is 1.3.0.
[...]
About every 4 "refresh", the memory used by my server increases by 96k, with nor release of memory. I checked the object number by watching len (gc.get_objects()), but it is stable. Any idea to help me to solve this problem, my server is expected to run continuously.
There was a leak in the _c_urlarg.c module in 1.3.0, which causes a string the size of the url to be leaked every request. This doesn't sound like it would account for the 96k per 4 requests to me, but perhaps the memory allocation strategy on QNX explains that. Either upgrade to Twisted 2.0 or newer, or grab _c_urlarg.c from that version or SVN (it's a one-line fix) and recompile it, or just delete the compiled module -- it'll fall back to the slower pure-python version in the standard library, but it won't leak. Let us know if that solves it, or at least helps. -Andrew.
thanks for help. I did not see any difference with the new _c_urlarg moduIe. I made then several trials and I found that Python 2.4.1 with Twisted 1.3.0 seems to be stable. It now runs for several days without leak. Something a little bit strange is that I tried Python 2.4.1 with Twisted 2.0.1 and TwistedWeb 0.5.0, but with this configuration the memory leak is very important. Yet with this config the objects number (length of gc.get_objects()) is growing with no collection, as I use exactly the same environment except Twisted version. Can you see a reason for that ? Andrew Bennetts <andrew-twisted@puzzling.org> Sent by: twisted-web-bounces@twistedmatrix.com 19/10/2005 16:05 Please respond to "Discussion of twisted.web, Nevow, and Woven" <twisted-web@twistedmatrix.com> To "Discussion of twisted.web, Nevow, and Woven" <twisted-web@twistedmatrix.com> cc Subject Re: [Twisted-web] Memory leak with twisted web server on QNX Security Level:? Internal On Wed, Oct 19, 2005 at 10:15:24AM +0200, denis.leborgne@rieter.com wrote:
Hi,
I use twisted as a very simple web server to serve python scripts. I now
have a
problem of memory leak on the server. I use Python 2.2.1 on QNX 6.3, Twisted version is 1.3.0. [...] About every 4 "refresh", the memory used by my server increases by 96k, with nor release of memory. I checked the object number by watching len (gc.get_objects()), but it is stable. Any idea to help me to solve this problem, my server is expected to run continuously.
There was a leak in the _c_urlarg.c module in 1.3.0, which causes a string the size of the url to be leaked every request. This doesn't sound like it would account for the 96k per 4 requests to me, but perhaps the memory allocation strategy on QNX explains that. Either upgrade to Twisted 2.0 or newer, or grab _c_urlarg.c from that version or SVN (it's a one-line fix) and recompile it, or just delete the compiled module -- it'll fall back to the slower pure-python version in the standard library, but it won't leak. Let us know if that solves it, or at least helps. -Andrew. _______________________________________________ Twisted-web mailing list Twisted-web@twistedmatrix.com http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-web
participants (2)
-
Andrew Bennetts -
denis.leborgne＠rieter.com