Exponential time increase in garbage collection
Sean McGrath
sean.mcgrath at propylon.com
Sun May 19 05:47:20 EDT 2002
At 14:38 18/05/2002 -0700, Neil Schemenauer wrote:
>Why you think the time is spent in GC? It sounds like the malloc
>implementation is trying to defragment memory on free(). The current GC
>has quadratic cost while allocating objects but it does not for
>deallocating objects. Can you try adding "gc.disable()" to the
>beginning of your program and see if the problem goes away?
I'm seeing curious figures (included below). Basically,
Python 1.5.2 exhibits a big difference in deletion time for
objects loaded via cPickle as opposed to pickle.py
enabling/disabling gc with Python 2.1.3 does not make any
appreciable difference. However, delete performance in
2.1.3 seems significantly worse than what I'm seeing
with Python 1.5.2 (30 seconds against 5 seconds).
I know that the ideal situation here would be a standalone
test case that I could post but I'm struggling with that
one on a number of fronts:-)
The figures I'm seeing:-
Python 2.1.3
load objects with pickle module:
gc enabled:
Load Time : 63
Delete Time : 31
gc disabled:
Load Tine : 63
Delete Time : 32
load objects with cPickle module:
gc enabled:
Load Time : 42
Delete Time : 32
gc disabled:
Load Time : 40
Delete Time : 32
Python 1.5.2:
load objects with pickle module:
Load Time : 80
Delete Time : 5 (!!!!!!!!!!!!!!!!)
load with cPickle module:
Load Time : 40
Delete Time : 43
Has the pickle format changed from 1.5.2 to 2.1.3? Is the fact that the pickles
where created with cPickle 1.5.2 skewing things I wonder?
regards,
Sean
More information about the Python-list
mailing list