[BangPypers] FW: how to relclaim memory

[Anand Balachandran Pillai]

Hi,

On Thu, Oct 23, 2008 at 4:26 PM, Sibtey Mehdi <sibteym at infotechsw.com> wrote:
> Python process is consuming near about 500 MB and 400MB others processes.
>
> One more problem I am facing is that if I pickle the heavy object, the
> python process consumes more then 700 MB. These 200 MB memory should be
> returned after the completion of the dumping but it doesn't return that's
> why I m thinking that there are some memory leaks.
>
Python pickle has memory issues. One sure shot way to write a memory hog
Python program is to keep pickling large objects once in a while, especially
if the object has cyclic references.

How big is the object you are pickling ? If your object is huge, i.e say
it is a container with more than 10,000 elements, you shouldn't try to
pickle it all at one shot. Or you can try reducing the pickling frequency.

If you are pickling GUI objects which are typically large, you can often
expect this kind of behaviour, especially objects used by higher level
APIs like pygame or wxPython, since the number of references they
hold is huge. Try a shallow pickle by writing your own __getstate__
function for the objects.

>
>
> I use the wmi to get the memory details.
>
> Memory Details..................
>
>
>
>         AvailableBytes = "75382784";
>
>         AvailableKBytes = "73616";
>
>         AvailableMBytes = "71";
>
>         CacheBytes = "42872832";
>
>         CacheBytesPeak = "296366080";
>
>         CacheFaultsPersec = 0;
>
>         CommitLimit = "2559868928";
>
>         CommittedBytes = "1345224704";
>
>         DemandZeroFaultsPersec = 58001;
>
>         FreeSystemPageTableEntries = 191814;
>
>         PageFaultsPersec = 63525;
>
>
>
> task manager Snaps
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
> Hi,
>
>
>
> On Thu, Oct 23, 2008 at 3:48 PM, Prashanth Ellina
>
> <prashanthellina at gmail.com> wrote:
>
>> Python 2.5 (unlike 2.4 and earlier versions) is expected to release memory
>
>> back to the OS when objects are de-referenced. Considering that I would
>
>> suspect that you are still holding references to python objects.
>
>>
>
>> Do you have an objects of user defined classed which define the __del__()
>
>> method? If yes, this would lead to an issue with for GC when collecting
>> for
>
>> circularly referenced objects.
>
>
>
> Still, it is quite unnatural for a single Python process to claim 900 MB
>
> of memory. Are you sure that the system memory usage is due to Python ?
>
> Did you check the task manager for this ?
>
>
>
> If you have cygwin installed, post the output of "top" command here.
>
>
>
>>
>
>> The gc module has some methods to help is figuring out what objects are
>
>> still referenced. That would be a good starting point to track down memory
>
>> issues.
>
>>
>
>> HTH,
>
>> Prashanth
>
>>
>
>> On Thu, Oct 23, 2008 at 3:30 PM, Sibtey Mehdi <sibteym at infotechsw.com>
>
>> wrote:
>
>>>
>
>>> I am using
>
>>>         Windows XP,
>
>>>         pythn 2.5, wxpython 2.8, pygame 1.8.
>
>>>
>
>>> Application over View:
>
>>> In my application I am creating a dialog box that shows a treeView items
>
>>> but
>
>>> on closing of that dialog box I delete the objects that contains the tree
>
>>> items. So I am trying to get that memory used by that tree items.
>
>>>
>
>>> Thanks,
>
>>> Sibtey
>
>>>
>
>>>
>
>>> Sibtey Mehdi <sibteym at infotechsw.com> wrote:
>
>>> > Hi
>
>>> >
>
>>> > My system is showing 900MB usage of memory, I have deleted some object
>
>>> (del
>
>>> > pythonObj) but the memory is not returned to os, I have also tried
>
>>> > gc.collect() and del (gc.garbage) but  still the system is showing
>>> > 900MB
>
>>> > usage.
>
>
>
> _______________________________________________
> BangPypers mailing list
> BangPypers at python.org
> http://mail.python.org/mailman/listinfo/bangpypers
>
>



-- 
-Anand