Exponential time increase in garbage collection

Sean McGrath sean.mcgrath at propylon.com
Sun May 19 05:47:20 EDT 2002


At 14:38 18/05/2002 -0700, Neil Schemenauer wrote:
>Why you think the time is spent in GC?  It sounds like the malloc
>implementation is trying to defragment memory on free().  The current GC
>has quadratic cost while allocating objects but it does not for
>deallocating objects.  Can you try adding "gc.disable()" to the
>beginning of your program and see if the problem goes away?

I'm seeing curious figures (included below). Basically,
Python 1.5.2 exhibits a big difference in deletion time for
objects loaded via cPickle as opposed to pickle.py

enabling/disabling gc with Python 2.1.3 does not make any
appreciable difference. However, delete performance in
2.1.3 seems significantly worse than what I'm seeing
with Python 1.5.2 (30 seconds against 5 seconds).

I know that the ideal situation here would be a standalone
test case that I could post but I'm struggling with that
one on a number of fronts:-)

The figures I'm seeing:-

Python 2.1.3
         load objects with pickle module:
                 gc enabled:
                         Load Time : 63
                         Delete Time : 31
                  gc disabled:
                         Load Tine : 63
                         Delete Time : 32
         load objects with cPickle module:
                 gc enabled:
                         Load Time : 42
                         Delete Time : 32
                  gc disabled:
                         Load Time : 40
                         Delete Time : 32
Python 1.5.2:
         load objects with pickle module:
                 Load Time : 80
                 Delete Time : 5                         (!!!!!!!!!!!!!!!!)
         load with cPickle module:
                 Load Time : 40
                 Delete Time : 43

Has the pickle format changed from 1.5.2 to 2.1.3? Is the fact that the pickles
where created with cPickle 1.5.2 skewing things I wonder?

regards,
Sean









More information about the Python-list mailing list