[Python-Dev] refleak hunting fun!

[Walter Dörwald]

Michael Hudson wrote:

> Prompted by finding my own refcounting leaks in test_descr, I'm now
> looking to spread the misery :-)
> 
> I've hacked regrtest (hacks attached) to do some very crude monitoring
> of sys.gettotalrefcount().
> 
> Here's the output of a recent run:
>
 > [...]
> test_codeccallbacks leaked 1107 references
 > [...]
> 
> Some however, seem to be real.  test_sax, test_socket and
> test_codeccallbacks seem to be the most alarming.  The TrackRef class
> Tim posted a link to is likely to be indispensable in hunting these.

The number for test_codeccallbacks really looks scary. Some of it is
the result of the callback registry, but there seem to be real leaks
here.

I've used the attached patch to unittest to get refcount diffs for
individual test methods. Here are the results for test_codeccallbacks:

   212 test_backslashescape
     1 test_badandgoodbackslashreplaceexceptions
     0 test_badandgoodignoreexceptions
     0 test_badandgoodreplaceexceptions
     0 test_badandgoodstrictexceptions
     0 test_badandgoodxmlcharrefreplaceexceptions
   271 test_badhandlerresults
     0 test_badregistercall
   153 test_callbacks
     3 test_charmapencode
   121 test_decodehelper
    26 test_encodehelper
   285 test_longstrings
     0 test_lookup
     9 test_relaxedutf8
     1 test_translatehelper
    11 test_unencodablereplacement
   265 test_unicodedecodeerror
   792 test_unicodeencodeerror
     0 test_unicodetranslateerror
    12 test_uninamereplace
     0 test_unknownhandler
    14 test_xmlcharnamereplace
     0 test_xmlcharrefreplace
     6 test_xmlcharrefvalues

Bye,
    Walter Dörwald

-------------- next part --------------
Index: Lib/unittest.py
===================================================================
RCS file: /cvsroot/python/python/dist/src/Lib/unittest.py,v
retrieving revision 1.24
diff -u -r1.24 unittest.py
--- Lib/unittest.py	13 Jul 2003 15:18:12 -0000	1.24
+++ Lib/unittest.py	14 Aug 2003 18:37:19 -0000
@@ -376,7 +376,13 @@
         for test in self._tests:
             if result.shouldStop:
                 break
+            import gc, sys
+            gc.collect()
+            rc1 = sys.gettotalrefcount()
             test(result)
+            gc.collect()
+            rc2 = sys.gettotalrefcount()
+            print >>sys.stderr, "%5d %s" % (rc2-rc1, test)
         return result
 
     def debug(self):