[Python-Dev] refleak hunting fun!

[Michael Hudson]

"Brett C." <bac at OCF.Berkeley.EDU> writes:

> Michael Hudson wrote:
>
>> Prompted by finding my own refcounting leaks in test_descr, I'm now
>> looking to spread the misery :-)
>> I've hacked regrtest (hacks attached) to do some very crude
>> monitoring
>> of sys.gettotalrefcount().
>
> <list of tests that might be leaking like the Titanic>
>
>> Some however, seem to be real.  test_sax, test_socket and
>> test_codeccallbacks seem to be the most alarming.  The TrackRef class
>> Tim posted a link to is likely to be indispensable in hunting these.
>> 
>
> <Brett's potentially ignorant newbie questions follow>
>
> Is there any reason not to add Michael's code to regrtest.py

Because it's at least something of a horrendous hack, at the moment.

> (or at least get it to the point of where it could be added)?

That's a better question <wink>.

> Finding leaks as part of running a test would be nice.

Certainly!  Then we might have found these *before* a major
release... though most of the leaks found so far are obscure enough
not to overly concern me.

> We could even have a set in regrtest.py that stored tests known to
> throw the test off (because of caching or whatever).

What my latest version does is run the test nine (!) times, and if
there seems to be a refcount abnormality, prints the refcount deltas
for the final four runs.  This helps spot some faux leaks
(e.g. test_mutants which does randomized testing).

Also, it's possible & desrible to do much finer testing on the more
structured tests (unittests & probably doctests, too).

> Obviously testing would be option that is off by default.

Yup, for several reasons...

> How about Tim's code class?

It should probably be floating around somewhere helpful, but I don't
think there's a way of automating this kind of thing.  If regrtest can
optionally point you at a problem, that's good enough, IMHO.

> Or is finding leaks that much of a random black art and there is not
> good way of doing it?
>
> Part of the reason I ask is I have had a patch sitting on SF for a
> while now that implements the skips.txt file idea that was brought up
> back when Martin was complaining about expected skips on certain
> platforms and such.  Is there a general view of not touching
> regrtest.py unless needed since breaking that would not be fun?

I don't think so.  Doing it a couple of weeks before 2.3 would have
been insane, but I don't think there's any special reason to avoid
enhancing regrtest.

The Python test suite is a bit untidy with the various methods of
testing (flat file, with test_main, unittests, doctests), but people
have been gradually unittest-ifying the tests, and I don't think
there's any reason to stop this or agitate to get it going faster.

Cheers,
mwh

-- 
  This is the fixed point problem again; since all some implementors
  do is implement the compiler and libraries for compiler writing, the
  language becomes good at writing compilers and not much else!
                                 -- Brian Rogoff, comp.lang.functional