[Python-Dev] Weak references: dereference notification

[Gustavo J. A. M. Carneiro]

On Wed, 2005-11-09 at 20:40 +0100, Ronald Oussoren wrote:
> On 9-nov-2005, at 18:52, Gustavo J. A. M. Carneiro wrote:
> 
> > Qua, 2005-11-09 às 09:23 -0800, Guido van Rossum escreveu:
> >>>> Gustavo J. A. M. Carneiro wrote:
> >>>>>   I have come across a situation where I find the current weak
> >>>>> references interface for extension types insufficient.
> >>>>>
> >>>>>   Currently you only have a tp_weaklistoffset slot, pointing to a
> >>>>> PyObject with weak references.  However, in my case[1] I  
> >>>>> _really_ need
> >>>>> to be notified when a weak reference is dereferenced.
> >>
> >> I find reading through the bug discussion a bit difficult to
> >> understand your use case. Could you explain it here? If you can't
> >> explain it you certainly won't get your problem solved! :-)
> >
> >   This is a typical PyObject wrapping C object (GObject) problem.   
> > Both
> > PyObject and GObject have independent reference counts.  For each
> > GObject there is at most one PyObject wrapper.
> >
> >   When the refcount on the wrapper drops to zero, tp_dealloc is  
> > called.
> > In tp_dealloc, and if the GObject refcount is > 1, I do something
> > slightly evil: I 'resurect' the PyObject (calling PyObject_Init),  
> > create
> > a weak reference to the GObject, and drop the "strong" reference.  I
> > call this a 'hibernation state'.
> 
> Why do you do that? The only reasons I can think of are that you hope  
> to gain
> some speed from this or that you want to support weak references to  
> the GObject.

  We want to support weak references to GObjects.  Mainly because that
support has always been there and we don't want/can't break API.  And it
does have some uses...

> 
> For what its worth, in PyObjC we don't support weak references to the  
> underlying
> Objective-C object and delete the proxy object when it is garbage  
> collected.
> Objective-C also has reference counts, we increase that in the  
> constructor for
> the proxy object and decrease it again in the destroctor.

  OK, but what if it is a subclass of a builtin type, with instance
variables?  What if the PyObject is GC'ed but the ObjC object remains
alive, and later you get a new reference to it?  Do you create a new
PyObject wrapper for it?  What happened to the instance variables?

  Our goal in wrapping GObject is that, once a Python wrapper for a
GObject instance is created, it never dies until the GObject dies too.
At the same time, once the python wrapper loses all references, it
should not stop keeping the GObject alive.

  What happens currently, which is what I'm trying to change, is that
there is a reference loop between PyObject and GObject, so that
deallocation only happens with the help of the cyclic GC.  But relying
on the GC for _everything_ causes annoying problems:

	1- The GC runs only once in a while, not soon enough if eg. you have an
image object with several megabytes;

	2- It makes it hard to debug reference counting bugs, as the symptom
only appears when the GC runs, far away from the code that cause the
problem in the first place;

	3- Generally the GC has a lot more work, since every PyGTK object needs
it, and a GUI app can have lots of PyGTK objects.

  Regards.

-- 
Gustavo J. A. M. Carneiro
<gjc at inescporto.pt> <gustavo at users.sourceforge.net>
The universe is always one step beyond logic