[Python-Dev] Real time behaviour of Pythons memory management; WAS: RE: Documentation about Python's GC,python-dev list messages referenced in Modules/gcmodule.c notreachable anymore

[Weber, Gregoire]

Garbage Collector findings

To understand pythons garbage collector better and to get
a picture about the runtime behaviour and performance 
I did some experiments:

The attached script allocates a lot of circularly self 
referencing lists. Then it instantiates a one item list.

I tuned the loop counter in such a way that the subsequent  
instantiation of the one item list triggers the garbage 
collector. The number of the circularly self referencing 
objects is around 90000.


Results (Pentium4 3GHz, Python 2.4.1 on cygwin/Win2k)
-----------------------------------------------------

 gc gen 1) | time msec 2) | unreachable 3)
-----------+--------------+----------------
 None      | 0.000        | All
 0         | 0.002        | All
 1         | 0.007        | All
 2         | 0.065        | All
 None      | 0.000        | None
 0         | 0.001        | None
 1         | 0.003        | None
 2         | 0.018        | None

--> Collecting a self referencing list costs about 
    0.7 usec (micro seconds)
--> Checking a non collectable self ref list costs 
    about 0.2 usec (micro seconds)

Legend:

1) the generation the garbage collector has been triggered 
   to collect (None means that GC wasn't triggered)
2) time for instantiating a list with one entry in msec
3) All: All of the circularly self referencing lists were 
        **unreachable** (thus got collected by the gc)
   None: None of the circularly self referencing lists were 
         **unreachable** (no garbage available to collect)


Questions
---------

1. Am I correct that in a system which instantiates a lot*) 
   of containerish objects without destructing them again the 
   GC may be triggered to evaluate all generations which may 
   be very costy (see above)?

2. In a system where only generation 0 and 1 get evaluated
   (because not so much object got instantiated without beeing 
   destructed) the costs are relatively low. Correct?

*) a lot means here: 
   more than ``threshold0 * threshold1 * threshold2`` objects, 
   Python 2.4 default values are: 700*10*10 = 70000


Gregoire




P.S.: Interestingely the counters found by experimenting 
      seem to depend on the platform (my platform: 
	Python 2.4.1, cygwin under Win2k, the counters aren't 
	valid for native Python 2.4.2 under Win2k).


-------------- next part --------------
A non-text attachment was scrubbed...
Name: gctest.py
Type: application/octet-stream
Size: 3936 bytes
Desc: gctest.py
Url : http://mail.python.org/pipermail/python-dev/attachments/20060104/a19e29c2/attachment.obj