[IronPython] IronPython with CherryPy through WSGI Memory issue

Thu Apr 1 23:44:02 CEST 2010

On Thu, Apr 1, 2010 at 8:08 PM, Jeff Hardy <jdhardy at gmail.com> wrote:
> On Thu, Apr 1, 2010 at 2:59 AM, Can Gencer <cgencer at gmail.com> wrote:
>> Hello,
>
> Hi Can,
> You are, to my knowledge, the first person besides me to use NWSGI for
> anything, so thank you!
>
>> Everything works after some tweaking done to CherryPy. I re-use the
>> application callable that is retrieved from the main script that is
>> compiled into CompiledCode . However the memory usage never seems to
>> go down, and goes up with every HTTP request, even if I force
>> GC.Collect() after every request.
>
> I haven't done any stress testing of NWSGI because my uses don't have
> any where near the traffic to need it :). It's quite possible that I'm
> holding to references to something (code objects maybe?) past there
> usefulness.
>
> Would it be possible to make the Python/CherryPy portion public? I'd
> like to take a look at it and try it out.
>
> Hopefully it's not hard to find, or I'll be reading Tess Ferrandez and
> learning how to use windbg again. Ugh.
>
> - Jeff

Hello,

The CherryPy code is very simple. I just have a "main.py" that gets
executed from the WSGI handler. All its doing is serving a static html
file that displays a lot of images which are stored under the
"/static" path. I am using CherryPy 3.1.2

---------------------------
import cherrypy

class HelloWorld:

    def index(self):
        return open('test.html').read()

    index.exposed = True

import os.path
tutconf = os.path.join(os.path.dirname(__file__), 'app.conf')
current_dir = os.path.dirname(os.path.abspath(__file__))
appconfig = {'/static' :
                    {
                       'tools.staticdir.dir':
os.path.join(current_dir, 'static'),
                       'tools.staticdir.on' : True
                    }
            }
cherrypy.config.update(tutconf)
application = cherrypy.tree.mount(HelloWorld(), "/", config=appconfig)
---------------------------

The app.conf looks like this (not sure if it's relevant, it was more
related to me testing a bunch of different stuff..)
---------------------------
[global]
tools.caching.on = False
tools.sessions.on = False
tools.sessions.timeout = 10
tools.encode.on: True
tools.encode.encoding: 'utf8'
log.screen = True
---------------------------
The html file I used for testing looks like this
---------------------------
<html>
<head>
<script type="text/JavaScript">
setTimeout("location.reload(true);", 5000);
</script>
</head>
<body>
<img src="/static/image.png"/>
<img src="/static/image2.png"/>
...
<img src="/static/image.png"/>
</body>
</html>
---------------------------
To get CherryPy working, I edited two files process/plugins.py, changed

import signal as _signal

to

try:
		import signal as _signal
except:
		_signal = None

as signal package doesn't exist in IronPyhton.

Another change I made was related to threading, as without this change
CherryPy doesn't work thread safe and the local thread storage seems
to get corrupted..

in __init__.py,  replaced
from threading import local as _local with from
cherrypy._cpthreadinglocal import local as _local

I don't know if the IronPython local module is buggy or this is
something related to CherryPy.. but this change seems to fix it.

It would be great if you can test this out with IIS and see if you get
the same results.. I don't know if IIS keeps one process in memory for
every request, but my web server is running as one process and reusing
the application callable

I will try Dino's suggestion about
ExceptionHelpers.DynamicStackFrames, is that a static class?