Tremendous slowdown due to garbage collection

Aaron Watters aaron.watters at
Mon Apr 14 22:27:42 CEST 2008

> A question often asked--and I am not a big a fan of these sorts of
> questions, but it is worth thinking about--of people who are creating
> very large data structures in Python is "Why are you doing that?"
> That is, you should consider whether some kind of database solution
> would be better.  You mention lots of dicts--it sounds like some
> balanced B-trees with disk loading on demand could be a good choice.

Well, probably because you can get better
than 100x improved performance
if you don't involve the disk and use clever in memory indexing.
BTW, I think the default behaviour of the gc is
pretty silly.  I tend to disable automatic gc and explicitly put in
collections when I know I'm done with some big operation these
   -- Aaron Watters


More information about the Python-list mailing list