I spent a little time poking around with a very simple test script (appended) that makes use of Neil's cycle gc patch. It measures the time it takes to instantiate and delete a cycle-producing instance 100,000 times (based upon a report in python-bugs). For vanilla 1.6a2 ("./configure" ; make") I got the following (last of six runs): memory usage @ start: 5560 clock: 1.33 elapsed: 1.34805500507 memory usage @ end: 18816 For 1.6a2 configured --with-cycle-gc ("./configure --with-cycle-gc ; make") I got the following (also the last of six runs): memory usage @ start: 5576 clock: 2.34 elapsed: 2.33785700798 memory usage @ end: 5576 I was very impressed with the memory usage, but dismayed at the increased execution time (about 75% more). Of course, this is a test script that is bound to show the potential performance penalty in the worst possible light. Maybe pystone will be more indicative of the performance hit we can expect from it. (There's a new one - pystone.py being a better indicator of true system performance than some other test script!) Plain: Pystone(1.1) time for 10000 passes = 1.76 This machine benchmarks at 5681.82 pystones/second --with-cycle-gc: Pystone(1.1) time for 10000 passes = 1.83 This machine benchmarks at 5464.48 pystones/second That suggests something more like a 4% performance penalty. I suspect that may be the lower limit. While I haven't looked at pystone recently, I doubt it creates any cyclic garbage. My next step will be to try it out on my development database server and see what happens. just-another-data-point-ly y'rs, -- Skip Montanaro, skip@mojam.com, http://www.mojam.com/, http://www.musi-cal.com/ "We have become ... the stewards of life's continuity on earth. We did not ask for this role... We may not be suited to it, but here we are." - Stephen Jay Gould class fred: def __init__(self): self.indirectFunc = self.theFunc def theFunc(self): return "blah" def test(): f = fred() del f if __name__ == "__main__": import os, time, sys sys.stdout.write("memory usage @ start: ") sys.stdout.flush() os.system("ps auxww | egrep method | egrep -v egrep | awk '{print $5}'") t = time.clock(), time.time() for x in xrange(100000): test() print "clock:", time.clock()-t[0], "elapsed:", time.time()-t[1] sys.stdout.write("memory usage @ end: ") sys.stdout.flush() os.system("ps auxww | egrep method | egrep -v egrep | awk '{print $5}'")
Skip, I don't have time today to look at your post in detail, but one thing you said rtiggered a pretty immediate response: We should worry more about the GC performance on examples like the one you posted, and less about pystone. pystone is a good benchmark for testing the overhead in cases when the garbage collector isn't invoked. There's so little memory allocation that nothing interesting happens. The GC patch ought to have no effect in this case. I suppose 4% is okay. The worry is about programs that allocate a lot of objects, even if they never create circular references. The compiler test is one example, because the parser tree consumes an enormous number of tuples and objects. I suspect big slowdowns like the one you posted just mean we don't know how to adjust the tuneable parameters. Jeremy
Jeremy> I don't have time today to look at your post in detail, but one Jeremy> thing you said rtiggered a pretty immediate response: We should Jeremy> worry more about the GC performance on examples like the one you Jeremy> posted, and less about pystone. I wasn't really worried about pystone, just looking for something quick that didn't generate cycles... ;-) Your suggestion is an excellent one. To try and account for the typical case, I modified the test slightly. I added a new class, ethyl: def foo(): pass class ethyl: def __init__(self): self.indirectFunc = foo def theFunc(self): return "blah" which performs essentially the same work at instantiation time as fred, but doesn't create a cycle. The time (in seconds) to create and delete 100,000 freds and ethyls with and without cycle-gc enabled looks like (last of six consecutive runs): ./python ./python-cycle-gc fred (cycle) 1.4 2.51 ethyl (no cycle) 1.39 1.55 The memory consumption is as you would expect - leaky when creating fred's without using cycle-gc, stable otherwise. So, the penalty for creating and deleting objects that don't contain cycles would appear to be reasonable (about 10-15%). As more experience is gained with it, I imagine it can be reduced further. Skip
participants (2)
-
Jeremy Hylton
-
Skip Montanaro