[Python-Dev] refleak hunting fun!

[Michael Hudson]

Prompted by finding my own refcounting leaks in test_descr, I'm now
looking to spread the misery :-)

I've hacked regrtest (hacks attached) to do some very crude monitoring
of sys.gettotalrefcount().

Here's the output of a recent run:

test_exceptions leaked 17 references
test_types leaked 22 references
test_class leaked 66 references
test_codeccallbacks leaked 1107 references
test_coercion leaked 14 references
test_compile leaked 69 references
test_complex leaked 2 references
test_cookie leaked 14 references
test_copy leaked 82 references
test_cpickle leaked 17 references
test_descr leaked 1 references
test_gc leaked 18 references
test_global leaked 14 references
test_hotshot leaked 111 references
test_logging leaked 82 references
test_minidom leaked 1 references
test_mutants leaked -452 references
test_new leaked 2 references
test_optparse leaked 408 references
test_os leaked 24 references
test_pkg leaked 7 references
test_pkgimport leaked 6 references
test_regex leaked 14 references
test_rotor leaked 14 references
test_sax leaked 1762 references
test_scope leaked 14 references
test_socket leaked 1140 references
test_sundry leaked 42 references
test_threadedtempfile leaked 81 references
test_threading leaked -24 references
test_time leaked -24 references
test_unicode leaked 29 references
test_urlparse leaked -70 references
test_xreadline leaked 14 references
test_grammar leaked 14 references

*some* of these are bogus -- caches in particular play merry hell with
trying to instrument this kind of thing -- and indeed I've already
found causes for some of the bogosities above but as my hacked
regrtest runs each test five times, runs are rraatthheerr ssllooww.

Some however, seem to be real.  test_sax, test_socket and
test_codeccallbacks seem to be the most alarming.  The TrackRef class
Tim posted a link to is likely to be indispensable in hunting these.

Some tests are a bit unhappy at being run repeatedly -- notably
test_threaded_import (which deadlocks) and all the doctest tests spew
stuff to stdout.  You'll want to add an '-x test_threaded_import' to
your command line.

I'd like to do some more work in aid of finding out exactly which test
cases in unittest-based are leaking the references.  Finding out which
of the flat-file style tests in test_types is leaking is not going to
be fun... (seems to be leaking a list of integers, oh dear hope that's
not my fault again :-/).

Cheers,
mwh

-------------- next part --------------
A non-text attachment was scrubbed...
Name: regr-hacks.diff
Type: text/x-patch
Size: 7406 bytes
Desc: not available
Url : http://mail.python.org/pipermail/python-dev/attachments/20030814/1918ff9f/regr-hacks.bin
-------------- next part --------------


-- 
  We've had a lot of problems going from glibc 2.0 to glibc 2.1.
  People claim binary compatibility.  Except for functions they
  don't like.                       -- Peter Van Eynde, comp.lang.lisp