[Python-bugs-list] [ python-Bugs-477963 ] Bad gc performance

[noreply@sourceforge.net]

Bugs item #477963, was opened at 2001-11-04 02:35
You can respond by visiting: 
http://sourceforge.net/tracker/?func=detail&atid=105470&aid=477963&group_id=5470

Category: Python Interpreter Core
Group: None
Status: Open
Resolution: None
Priority: 5
Submitted By: Roeland Rengelink (rengelink)
Assigned to: Nobody/Anonymous (nobody)
Summary: Bad gc performance

Initial Comment:
The attached program demonstrates bad (maybe O(n**2))
performance of the gc when collecting garbage

The program first creates a dictionary d with N
7-character keys referring a unique integer (2N
objects) the dictionary is then set to None. The
program times the creation of the dictionary (in
micro-sec/item) and then the time it takes to execute
d=None also expressed in (usec/item). Results are the
following:

(Python2.2b1, default conf, Linux 2.2.14, Suse 6.2)

~>python timing.py 
size:   10000, creation: 33.67 (usec/elem),
destruction:  1.59 (usec/elem)
size:   20000, creation: 33.18 (usec/elem),
destruction:  1.67 (usec/elem)
size:   50000, creation: 35.48 (usec/elem),
destruction:  1.83 (usec/elem)
size:  100000, creation: 34.81 (usec/elem),
destruction:  2.01 (usec/elem)
size:  200000, creation: 34.63 (usec/elem),
destruction:  2.50 (usec/elem)
size:  400000, creation: 35.12 (usec/elem),
destruction:  3.89 (usec/elem)
size:  600000, creation: 34.16 (usec/elem),
destruction:  6.07 (usec/elem)
size:  800000, creation: 34.90 (usec/elem),
destruction:  8.10 (usec/elem)
size: 1000000, creation: 34.53 (usec/elem),
destruction: 11.14 (usec/elem)

This test does not exceed main memory (128M) on my
machine and CPU utilization remains 95+% throughout. If
the test is extended to 2000000 objects on my machine
(requires VM) gc leads to an extended period of
disk-thrashing (CPU at 1%) requiring many minutes to
clean up the garbage.

A similar problem was reported by Fred in

http://mail.python.org/pipermail/python-list/2001-November/070667.html

I'm not sure if this is a bug. My naive expectation is
that cleanup should be O(n), but then I don't really
kmow what the gc is up to.

Let me know if you need more info

Roeland

----------------------------------------------------------------------

You can respond by visiting: 
http://sourceforge.net/tracker/?func=detail&atid=105470&aid=477963&group_id=5470