[Python-Dev] Object finalization for local (ie function) scopes

[Oliver Schoenborn]

----- Original Message ----- 
From: "Martin v. Löwis" <martin at v.loewis.de>


> Oliver Schoenborn wrote:
> > This is a module I made to test a concept of finalization *not* based on
> > __del__. I'm curious to find out if the techniques look like they could
be
> > implemented directly in the Python interpreter to lessen the burden on
the
> > user, which is why I post this here.
>
> Yes, it could be implemented directly on the user. However, I notice
> that the algorithm you implement has nothing to do with finalization.
> ...
> In summary: to avoid confusion, you should not use the term
> "finalization" when talking about the mechanisms you have implemented
> in the module.

Granted, it's not finalization in the technical sense but it is in the
logical sense: when the function returns, the try/finally calls a method
which must assume that the object will no longer be used after that (i.e. no
other references will be keeping the object alive past the function).

> Python does provide finalization, and in a reliable way: the finalizer
> is called before the object goes away, guaranteed.

The python docs are a bit confusing to me in this regard, so allow me to
quote a few section:

  From Python 2.3 docs:

      "An implementation is allowed to postpone garbage collection
      or omit it altogether -- it is a matter of implementation
      quality how garbage collection is implemented, as long as no
      objects are collected that are still reachable. (Implementation
      note: the current implementation uses a reference-counting
      scheme with (optional) delayed detection of cyclically linked
      garbage, which collects most objects as soon as they become
      unreachable, but is not guaranteed to collect garbage containing
      circular references. ...)"

>From Python 2.3 docs, section 3.1 of Library Ref:

    "Some objects contain references to external resources such
    as open files or windows. It is understood that these resources
    are freed when the object is garbage collected, but since garbage
collection is
    not guaranteed to happen, such objects also provide an explicit
    way to release the external resource, usually a close() method.
    Programs are strongly recommended to explicitly close such
    objects. The 'try...finally' statement provides a convenient way
    to do this."

>From section 3.3.1:

      __del__( self)

Called when the instance is about to be destroyed. .... It is not
guaranteed that __del__() methods are called for objects that
still exist when the interpreter exits.  ...x.__del__() ... is only
called when x's reference count reaches zero.

And finally, from 1.10 of "Extending and Embedding...":

"While Python uses the traditional reference counting implementation, it
also offers a cycle detector that works to detect reference cycles. This
allows applications to not worry about creating direct or indirect circular
references"

This seems to indicate that there are several situations to keep in mind:

1) __del__ is called automatically when reference count goes to zero; this
is independent of the presence of gc; however, standard doesn't garantee
that __del__ gets called for objects still alive when interpreter exits, so
for critical pieces of code (like releasing mutex locks), this could be
dangerous; plus the object is kept alive in the case of an exception, so
can't rely on its __del__ being called if exception is not caught.

2) the (optional) garbage collector handles the cases where reference count
never goes to zero due to cyclical refs involving at least one class with a
__del__ defined; however it is implementation-defined whether this exists
and if it does, when it does its job; so classes that have a __del__ are
unlikely to get freed by the gc if they are involved in a cyclical ref.

So it seems that having a __del__ is ok *most* of the time, and that if you
find a mem leak, verify if you have cyclical refs, and modify algorithm to
break cycle because gc won't be able to call __del__; however if an
exception gets thrown, you can't rely on __del__ getting called. Since
exceptions are so common, I have to conclude that you can never rely on
__del__ to free resources. Where am I wrong in this reasoning?

> What you want is a function that is called when the object goes of of
> some scope. This is something completely different, as the object may
> life long and prosper even after it goes out of scope.

> You have an assertion
>
>      assert sys.getrefcount(self) == 5
>
> It is very easy to trigger that assertion, so I'm uncertain what
> the intended meaning of that assertion is.

Was useful only during testing and has since been removed.

Thanks for the feedback!

Oliver