[Python-Dev] Weakref design questions

[David Abrahams]

Brian Quinlan <brian@sweetapp.com> writes:

> David Abrahams:
> > What's the point of the proxy? You never use it for anything. I guess
> > you must've meant:
> 
> >     python_function(proxy)
> 
> > above.
> 
> <blush> Nice catch.
> 
> > Hum. In Boost.Python the default is to copy the C++ object when
> > passing it to a Python callback, though it's possible for the user
> > to explicitly say "just build a Python object around a reference to 
> > the C++ object -- Python code beware of lifetime issues". I guess I 
> > like your idea, though, as a safer alternative for non-copyable or 
> > for very expensive-to-copy C++ objects.
> 
> DOMs, the data structures in question, can be obscenely expensive to
> copy.

This may not be possible for you, but in C++ That would generally be a
"handle" class with reference semantics (using a reference-counted
smart pointer, for example) so that "copying" the DOM really just
amounted to bumping a reference count somewhere.

> And I firmly believe that Python code should never be able to cause a
> crash.

Me too, but some of my users insist on being able to trade safety for
performance.

> [nice demonstration deleted]
> > How does this differ from what's happening for you?
> 
> When you created the bound method p.append, you caused the reference
> count of p to be increased. 

I think I caused the reference count of l to be increased. AFAIK, p
isn't keeping the list alive.

> In my case, since Python is not capable of altering the lifetime of p
> (it's dead with the C library says that it is dead), this is a bad
> thing.

Yeah, I see that now.

> So I proxy the bound methods and kill them when p dies.

I don't quite understand what "proxy the bound methods" means.

> > Hmm, it kinda seems like you want to prune some of them before the
> > user's done with them.
> 
> I actually presented two different desires at once and know I'm paying
> the price. 
> 
> There are two reasons that I might want to delete a proxied object:
> 
> 1. Its lifetime is over and I don't want anyone to access it

OK.

> 2. There are no proxy objects that refer to it, making it inaccessible
> (since I hold the only actual reference)
> 
> Reason #1 is why I could like builtin methods to support weakrefs.
> Reason #2 is why I would like a callback when the last weakref dies.

So you want a weakrefref. Well, I think this problem can be solved by
applying the Fundamental Theorem of Software Engineering: apply an
extra level of indirection. But Guido's approach of having the wrapper
methods check for a null DOM* seems like a reasonable one to
me. Incidentally, that's what you automatically get from Boost.Python
when your object is held by std::auto_ptr<T>, which is a smart pointer
class that can release ownership. I think I want to do something
similar for boost::weak_ptr, but I haven't gotten around to it yet.

> > Don't users expect objects to stay alive when they hold references?
> 
> Probably. But that is not always an option.

That's clear to me now. Very interesting thread; thanks for posting it
here!

-- 
                    David Abrahams
dave@boost-consulting.com * http://www.boost-consulting.com

Building C/C++ Extensions for Python: Dec 9-11, Austin, TX
http://www.enthought.com/training/building_extensions.html