[Tutor] memory management

[David Ascher]

On Fri, 16 Apr 1999, Arne Mueller wrote:

> I can traverse the tree using the following loops
> 
> for i in root.list1:
>         for k in keys():
> 	           for l in i.hits[l].list2:
> 		  	action(l)
>                 	l.remove 
>         	   del i.dictionary.[k]
>         i.remove

Several comments:

1) Taking the code above at face value, you're forgetting to call the
   remove method as your code doesn't have the required ()'s -- it
   should read 
			l.remove() # note the ()!
   and
			i.remove() # same here.

   That wouldn't help, though, since remove is a method of lists which
   takes an element as argument.  

2) You have a line which reads:

         for l in i.hits[l].list2:

   but l isn't defined the first time around, at least not in the snippet
   you gave.
  
3) Similarly, you call the "keys()" function which isn't defined.
 
4) Similarly, the line:

       	   del i.dictionary.[k]

   is not syntactically valid Python.  


When posting code, please post working code, not pseudocode -- Python is
so simple and clear, there's no need to make up a new syntax.

   
With a lot of guess work, I'd say that you need something like:

  for outer_index in range(len(outer_list)):
    outer_element = outer_list[outer_index]
    for key in outer_element.keys():
      for inner_index in range(len(outer_element[key])):
        action(outer_element[key][inner_index])
        del outer_element[key][inner_index]   # delete inner element
      del outer_element[key]                  # delete inner list
    del outer_element                         # delete dictionary
    del outer_list[outer_index]               # delete outer element

This gives you maximum control over the deletion of each reference -- it
is probably overkill.  Note that it does not guarantee that the memory is
freed -- Python frees memory when no references to an object exist.  So,
if you have a memory leak, the problem is that you have some references to
an object lying around somewhere -- usually in a dictionary or a list.

Christian's comment about rethinking the code is a valid one -- it might
save you time to think a bit about whether you really need to have all 100
Mb in memory at the same time, or whether you can pipeline the process
somewhat.  Without knowing more about the specifics, it's hard to know how
to help.

--david ascher