[Python-Dev] Patch for Weak References for Functions

[Phil Thompson]

"Fred L. Drake, Jr." wrote:
> 
> Phil Thompson writes:
>  > Any chance of the attached small patch be applied to enable weak
>  > references to functions?
>  >
>  > It's particularly useful for lambda functions and closes the "very last
>  > loophole where a programmer can cause a PyQt script to seg fault" :)
> 
> Phil,
>   Can you explain how this would help with the memory issues?  I'd
> like to have a better idea of how this would make things work right.
> Are there issues with the cyclic GC with respect to the Qt/KDE
> bindings?

Ok, some background...

Qt implements a component model for its widgets. You build applications
by sub-classing the standard widgets and then "connect" them together.
Connections are made between signals and slots - both are defined as
class methods. Connections perform the same function as callbacks in
more traditional GUI toolkits like Xt. Signals/slots have the advantage
of being type safe and the resulting component model is very powerful -
it encourages class designers to build functionally rich component
interfaces.

PyQt supports this model. It also allows slots to be any Python callable
object - usually a class method. You create a connection between a
signal and slot using the "connect" method of the QObject class (from
which all objects that have signals or slots are derived). connect()
*does not* increment the reference count of a slot that is a Python
callable object. This is a design decision - earlier versions did do
this but it almost always results in circular reference counts. The
downside is that, if the slot object no longer exists when the signal is
emitted (because the programmer has forgotten to keep a reference to the
class instance alive) then the usual result is a seg fault. These days,
this is the only way a PyQt programmer can cause a seg fault with bad
code (famous last words!). This accounts for 95% of PyQt programmer's
problem reports.

With Python v2.1, connect() creates a weak reference to the Python
callable slot. When the signal is emitted, PyQt (actually it's SIP)
finds out that the callable has disappeared and takes care not to cause
the seg fault. The problem is that v2.1 only implements weak references
for class instance methods - not for all callables.

Most of the time callables other than instance methods are fairly fixed
- they are unlikely to disappear - not many scripts start deleting
function definitions. The exception, however, is lambda functions. It is
sometimes convenient to define a slot as a lambda function in order to
bind an extra parameter to the slot. Obviously lambda functions are much
more transient than regular functions - a PyQt programmer can easily
forget to make sure a reference to the lambda function stays alive. The
patch I proposed gives the PyQt programmer the same protection for
lambda functions as Python v2.1 gives them for class instance methods.

To be honest, I don't see why weak references have been implemented as a
bolt-on module that only supports one particular object type. The thing
I most like about the Python implementation is how consistent it is.
Weak references should be implemented for every object type - even for
None - you never know when it might come in useful.

As far as cyclic GC is concerned - I've ignored it completely, nobody
has made any complaints - so it either works without any problems, or
none of my user base is using it.

Phil