Can someone explain this weakref behavior?
David MacQuigg
dmq at gain.com
Fri Jun 11 17:15:50 EDT 2004
On Fri, 11 Jun 2004 15:38:29 -0400, "Tim Peters" <tim.one at comcast.net>
wrote:
>[Tim Peters]
>>> That will pass under CPython today, but there's no general guarantee
>>> about exactly when a weak dict will notice that keys (or values) have
>>> become unreachable by strong references.
>
>[David MacQuigg]
>> OUCH!! We just built a module that uses weak references to keep a
>> "robust count" of instances in various classes. The claim is that this
>> is more robust than simply incrementing and decrementing class variables
>> using __init__ and __del__. The module seems to be working OK,
>> immediately deleting the weak reference as soon as all references to the
>> corresponding instance are deleted.
>
>Then it must be the case that you're running CPython, and that these
>instances aren't involved in cycles. Because CPython primarily uses
>reference-counting to recycle garbage, its behavior is predictable in the
>absence of cycles.
I'm not worried about cyclic references, but it is something to keep
in mind.
>CPython's use of reference counting is an implementation detail, and that
>internal weakref lists are traversed "immediately" upon an object's refcount
>reaching 0 is also an implementation detail. Nothing in the language
>definition guarantees these behaviors.
>
>> If I understand you correctly, there is some chance that a future
>> implementation of Python may have the weak references "out-of-sync" with
>> the actual count of live instances. Is that a remote possibility, or
>> something quite likely to occur?
>
>Well, it's been the case for a long time in JPython. I judge the odds of it
>changing in CPython as slim. I personally wouldn't worry about it ever
>changing in CPython. If a PyPy- or Parrot-based implementation of Python
>takes off, behavior will depend on its implementation details.
>
>> I have to decide now whether to rip out some risky code.
>>
>> Is there a good way to track the count of instances?
>
>If you want it enough, you can build Python in a mode that tracks this
>automatically (see the discussion of COUNT_ALLOCS in Misc/SpecialBuilds.txt
>-- for each distinct type object, the total # of allocations, # of
>deallocations, and highwater mark (max(#alloc - #dealloc) over time) are
>maintained in a COUNT_ALLOCS build).
>
>> If not, would it make sense to request a guarantee on the current
>> behavior of weak references? Maybe it could be an option, assuming there
>> is some performance penalty, an option to be used when accuracy is more
>> important than speed.
>
>You can request anything you can dream up <wink>. If it's something your
>business needs, the only way to guarantee it is to get involved in Python
>development deeply enough so that, if worse comes to worse, you can maintain
>your own Python implementation. That's unreasonably paranoid in my
>estimation, but it's a judgment call.
Seems like we could do this more easily with a function that lists
instances, like __subclasses__() does with subclasses. This doesn't
have to be efficient, just reliable. So when I call
cls.__instances__(), I get a current list of all instances in the
class.
Maybe we could implement this function using weak references. If I
understand the problem with weak references, we could have a
WeakValueDictionary with references to objects that actually have a
refcount of zero. There may be too many entries in the dictionary,
but never too few. In that case, maybe I could just loop over every
item in my WeakValueDictionary, and ignore any with a refcount of
zero.
def _getInstances(cls):
d1 = cls.__dict__.get('_instances' , {})
d2 = {}
for key in d1:
if sys.getrefcount(d1[key]) > 0:
d2[key] = d1[key]
return d2
_getInstances = staticmethod(_getInstances)
I'm making some assumptions here that may not be valid, like
sys.getrefcount() for a particular object really will be zero
immediately after all normal references to it are gone. i.e. we don't
have any temporary "out-of-sync" problems like with the weak
references themselves.
Does this seem like a safe strategy?
-- Dave
More information about the Python-list
mailing list