creating garbage collectable objects (caching objects)

[Dave Angel]

Gabriel Genellina wrote:
> <div class="moz-text-flowed" style="font-family: -moz-fixed">En Mon, 
> 29 Jun 2009 08:01:20 -0300, Dave Angel <davea at ieee.org> escribió:
>> News123 wrote:
>
>>> What I was more concerned is a group of output images depending on TWO
>>> or more input images.
>>>
>>> Depending on the platform (and the images) I might not be able to
>>> preload all two (or more images)
>>>
>>> So,  as CPython's garbage collection takes always place immediately,
>>> then I'd like to pursue something else.
>>> I can create a cache, which caches input files as long as python leaves
>>> at least n MB available for the rest of the system.
>
>> As I said earlier, I think weakref is probably what you need.  A 
>> weakref is still a reference from the point of view of the 
>> ref-counting, but not from the point of view of the garbage 
>> collector.  Have you read the help on weakref module?  In particular, 
>> did you read Pep 0205?   http://www.python.org/dev/peps/pep-0205/
>
> You've misunderstood something. A weakref is NOT "a reference from the 
> point of view of the ref-counting", it adds zero to the reference 
> count. When the last "real" reference to some object is lost, the 
> object is destroyed, even if there exist weak references to it. That's 
> the whole point of a weak reference. The garbage collector isn't 
> directly related.
>
> py> from sys import getrefcount as rc
> py> class X(object): pass
> ...
> py> x=X()
> py> rc(x)
> 2
> py> y=x
> py> rc(x)
> 3
> py> import weakref
> py> r=weakref.ref(x)
> py> r
> <weakref at 00BE56C0; to 'X' at 00BE4F30>
> py> rc(x)
> 3
> py> del y
> py> rc(x)
> 2
> py> del x
> py> r
> <weakref at 00BE56C0; dead>
>
> (remember that getrefcount -as any function- holds a temporary 
> reference to its argument, so the number it returns is one more than 
> the expected value)
>
>> Object cache is one of the two reasons for the weakref module.
>
> ...when you don't want the object to stay artificially alive just 
> because it's referenced in the cache. But the OP wants a different 
> behavior, it seems. A standard dictionary where images are removed 
> when they're no more needed (or a memory restriction is fired).
>
Thanks for correcting me.  As I said earlier, I have no experience with 
weakref.  The help and the PEP did sound to me like it would work for 
his needs.

So how about adding an attribute in the large object that refers to the 
object iself?.  Then the ref count will never go to zero, but it can be 
freed by the gc.  Also store the ref in a WeakValueDictionary, and you 
can find the object without blocking its gc.

And no, I haven't tried it, and wouldn't unless a machine had nothing 
important running on it.  Clearly, the gc might not be able to keep up 
with this kind of abuse.  But if gc is triggered by any attempt to make 
too-large an object, it might work.

DaveA