Python server locks up

[M.-A. Lemburg]

Zac Burns wrote:
> I have a server running Python 2.6x64 which after running for about a
> month decides to lock up and become unresponsive to all threads for
> several minutes at a time. While it is locked up Python proceeds to
> consume large amounts of continually increasing memory.
> 
> The basic function of the server is to serve a large dictionary -
> somewhat like a database. I have a couple theories as to why it locks
> up, but I'm not sure how to test them.
> 
> Theories:
>    Python is resizing the large dictionary
>    Python is garbage collecting
> 
> How would you suggest to figure out what is the problem?

Enable process core dumps and then kill the process.

Using a debugger you should be able to find out what caused the hang.

If you're serving a dictionary, I'd suggest using a different
approach: keep the dictionary on disk instead of in memory.

mxBeeBase will let you do that quite easily and is fast at
it as well:

http://www.egenix.com/products/python/mxBase/mxBeeBase/

It includes a tunable caching mechanism to keep the most often
requested data in memory.

-- 
Marc-Andre Lemburg
eGenix.com

Professional Python Services directly from the Source  (#1, Sep 14 2009)
>>> Python/Zope Consulting and Support ...        http://www.egenix.com/
>>> mxODBC.Zope.Database.Adapter ...             http://zope.egenix.com/
>>> mxODBC, mxDateTime, mxTextTools ...        http://python.egenix.com/
________________________________________________________________________

::: Try our new mxODBC.Connect Python Database Interface for free ! ::::


   eGenix.com Software, Skills and Services GmbH  Pastor-Loeh-Str.48
    D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg
           Registered at Amtsgericht Duesseldorf: HRB 46611
               http://www.egenix.com/company/contact/