[Python-Dev] extremely slow exit for program having huge (45G) dict (python 2.5.2)

Mike Coleman tutufan at gmail.com
Sat Dec 20 00:29:38 CET 2008


I have a program that creates a huge (45GB) defaultdict.  (The keys
are short strings, the values are short lists of pairs (string, int).)
 Nothing but possibly the strings and ints is shared.

The program takes around 10 minutes to run, but longer than 20 minutes
to exit (I gave up at that point).  That is, after executing the final
statement (a print), it is apparently spending a huge amount of time
cleaning up before exiting.  I haven't installed any exit handlers or
anything like that, all files are already closed and stdout/stderr
flushed, and there's nothing special going on.  I have done
'gc.disable()' for performance (which is hideous without it)--I have
no reason to think there are any loops.

Currently I am working around this by doing an os._exit(), which is
immediate, but this seems like a bit of hack.  Is this something that
needs fixing, or that has already been fixed?

Mike


More information about the Python-Dev mailing list