[Python-Dev] refleak hunting fun!
mwh at python.net
Fri Aug 15 13:06:39 EDT 2003
"Brett C." <bac at OCF.Berkeley.EDU> writes:
> Michael Hudson wrote:
>> Prompted by finding my own refcounting leaks in test_descr, I'm now
>> looking to spread the misery :-)
>> I've hacked regrtest (hacks attached) to do some very crude
>> of sys.gettotalrefcount().
> <list of tests that might be leaking like the Titanic>
>> Some however, seem to be real. test_sax, test_socket and
>> test_codeccallbacks seem to be the most alarming. The TrackRef class
>> Tim posted a link to is likely to be indispensable in hunting these.
> <Brett's potentially ignorant newbie questions follow>
> Is there any reason not to add Michael's code to regrtest.py
Because it's at least something of a horrendous hack, at the moment.
> (or at least get it to the point of where it could be added)?
That's a better question <wink>.
> Finding leaks as part of running a test would be nice.
Certainly! Then we might have found these *before* a major
release... though most of the leaks found so far are obscure enough
not to overly concern me.
> We could even have a set in regrtest.py that stored tests known to
> throw the test off (because of caching or whatever).
What my latest version does is run the test nine (!) times, and if
there seems to be a refcount abnormality, prints the refcount deltas
for the final four runs. This helps spot some faux leaks
(e.g. test_mutants which does randomized testing).
Also, it's possible & desrible to do much finer testing on the more
structured tests (unittests & probably doctests, too).
> Obviously testing would be option that is off by default.
Yup, for several reasons...
> How about Tim's code class?
It should probably be floating around somewhere helpful, but I don't
think there's a way of automating this kind of thing. If regrtest can
optionally point you at a problem, that's good enough, IMHO.
> Or is finding leaks that much of a random black art and there is not
> good way of doing it?
> Part of the reason I ask is I have had a patch sitting on SF for a
> while now that implements the skips.txt file idea that was brought up
> back when Martin was complaining about expected skips on certain
> platforms and such. Is there a general view of not touching
> regrtest.py unless needed since breaking that would not be fun?
I don't think so. Doing it a couple of weeks before 2.3 would have
been insane, but I don't think there's any special reason to avoid
The Python test suite is a bit untidy with the various methods of
testing (flat file, with test_main, unittests, doctests), but people
have been gradually unittest-ifying the tests, and I don't think
there's any reason to stop this or agitate to get it going faster.
This is the fixed point problem again; since all some implementors
do is implement the compiler and libraries for compiler writing, the
language becomes good at writing compilers and not much else!
-- Brian Rogoff, comp.lang.functional
More information about the Python-Dev