[Chicago] Out of Memory: Killed Process: on CentOS

[Ishmael Rufus]

I'm assuming this is a Linux based server.
How much memory does the process acquire before the server stops responding?

On Mon, Apr 27, 2009 at 10:16 AM, Brian Ray <bray at sent.com> wrote:

> Hello:
>
> I am trying to tackle an issue I am having with httpd running an app under
> mod_python not being able to kill enough processes (to free memory) quickly
> enough to remain stable.  The server ends up in a state where it stops
> responding.
>
> The server has several Gig's of memory, but the requests it tries to
> process are really huge.  It run's fine if the requests are small.
> Basically, the requests contain XML that is parsed by ElementTree (C
> version) and converts that data to an internal database format.  Somewhere
> in the parsing of that XML is where is seems to eat the most memory, but
> writing to the database uses some, as well.
>
> What is the best ways to manage this scenario?  Specifically, what should I
> try?  One thing I thought about doing is repetitively calling "free -m"  or
> "ps .." and parse the results to see if something has to be done.  I am ok
> with the idea of telling the caller they need to try again later or try
> smaller chunks--and I would prefer this than sending no response at all.
>
> Should I be looking at my apache configuration?  I tried setting
> StartServers, MinSpareServers, MaxSpareServers, and MaxClients to low
> numbers, like 3.  Overall it seems to help but, I am not sure it will if I
> get a really huge request.
>
> Should I run the parser outside of Apache?  I was currious if Python by
> itself (not embedded into Apache) would run more efficently in this
> situation.
>
> Is there some other way to clear memory when running large requests?
>
>
> If you help me out with a good solution, I will be open to presenting (or
> talking you into presenting) at a future ChiPy meeting on what was done to
> tackle this issue.
>
>
> Regards,
> Brian Ray
>
>
>
> _______________________________________________
> Chicago mailing list
> Chicago at python.org
> http://mail.python.org/mailman/listinfo/chicago
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/chicago/attachments/20090427/e4da378b/attachment.htm>