[Twisted-Python] twisted eat the memory
I wrote web application on twisted and my application read from ulogd daemon and push the output to the client browser using AJAX .also I have another application displaying ethernets traffic using AJAX. but after some minutes the twisted process in the server eat the memory and the OOM-killer of the linux will kill the twisted process. are there any solution to make the twisted release the memory . note: my ram is 256mb -- Muaaz Osman Hussain IT Researcher
What version of Python are you using? 2.4 and below have problems releasing memory back to the OS. Since Twisted apps are normally long running Python daemons they can be more prone to memory leaks. You need to be aware of how objects are garbage collected in Python and rework your code if necessary. Twisted itself should not be the source of problem. An interesting article on the subject: http://evanjones.ca/python-memory.html -Cary On Mon, Dec 15, 2008 at 12:31 PM, Muaaz Hussain <muaazhussain128@gmail.com> wrote:
-- 01100011 01100001 01110010 01111001
On Mon, 15 Dec 2008 20:31:39 +0300, Muaaz Hussain <muaazhussain128@gmail.com> wrote:
This isn't a feature of Twisted, so there's no simple flag you can pass or alternate API you can use to avoid it. Perhaps you can share a minimal, self-contained example program which produces this behavior and someone can point out what the cause of the memory use is. Make sure the example is short (does not contain code unrelated to the high memory usage) and can be run to produce the problem. Jean-Paul
Am Mon, 15 Dec 2008 20:31:39 +0300 schrieb "Muaaz Hussain" <muaazhussain128@gmail.com>:
Well, CPython uses a dual garbage collection policy (reference counting, and cycle breaking GC), and there are at least some cases where garbage can leak. (cycles with __del__ methods defined.) But in a different way, that only helps you manage memory, but in the end you as the developer need to decide which objects you still need to forget about the object before Python can release the memory. Another point is that under Unix, processes tend to keep memory they have allocated to themselves, so if your process needs say 300MB at start-up and releases 270MB you need to check if these really end up available for the OS. But as exarkun pointed out, that's a nontrivial problem, and without reducing it to a small and runnable example that shows your problem, the list won't be able to help you much. (If you have such general python questions, you might consider posting also to tutor@python.org, which tends to be slightly more considerate to newbies.) Andreas
What version of Python are you using? 2.4 and below have problems releasing memory back to the OS. Since Twisted apps are normally long running Python daemons they can be more prone to memory leaks. You need to be aware of how objects are garbage collected in Python and rework your code if necessary. Twisted itself should not be the source of problem. An interesting article on the subject: http://evanjones.ca/python-memory.html -Cary On Mon, Dec 15, 2008 at 12:31 PM, Muaaz Hussain <muaazhussain128@gmail.com> wrote:
-- 01100011 01100001 01110010 01111001
On Mon, 15 Dec 2008 20:31:39 +0300, Muaaz Hussain <muaazhussain128@gmail.com> wrote:
This isn't a feature of Twisted, so there's no simple flag you can pass or alternate API you can use to avoid it. Perhaps you can share a minimal, self-contained example program which produces this behavior and someone can point out what the cause of the memory use is. Make sure the example is short (does not contain code unrelated to the high memory usage) and can be run to produce the problem. Jean-Paul
Am Mon, 15 Dec 2008 20:31:39 +0300 schrieb "Muaaz Hussain" <muaazhussain128@gmail.com>:
Well, CPython uses a dual garbage collection policy (reference counting, and cycle breaking GC), and there are at least some cases where garbage can leak. (cycles with __del__ methods defined.) But in a different way, that only helps you manage memory, but in the end you as the developer need to decide which objects you still need to forget about the object before Python can release the memory. Another point is that under Unix, processes tend to keep memory they have allocated to themselves, so if your process needs say 300MB at start-up and releases 270MB you need to check if these really end up available for the OS. But as exarkun pointed out, that's a nontrivial problem, and without reducing it to a small and runnable example that shows your problem, the list won't be able to help you much. (If you have such general python questions, you might consider posting also to tutor@python.org, which tends to be slightly more considerate to newbies.) Andreas
participants (4)
-
Andreas Kostyrka -
Cary Hull -
Jean-Paul Calderone -
Muaaz Hussain