Why no open(f, "w").write()?

[Delaney, Timothy]

> From: Andreas Kostyrka [mailto:andreas at kostyrka.priv.at]
>
> > Me ...
> >
> > __files = []
> >
> > def open (path, mode):
> >     f = __open(path, mode)
> >     __files.append(f)
> >     return f
> >
> > or
> >
> > class file:
> >
> >     __files = []
> >
> >     def __init__(self, path, mode):
> >         __files.append(self)
> >
> > Whilst I know this is not the implementation, there is nothing that
> > prevents it from being like this. So you get back an object 
> that has one
> > more reference count than expected. It never gets collected.

> Well, then the implementation is broken. If you allow the 
> implementation
> to collect additional references at random places, you can prove quite
> easily that nothing is safe :)

Not at all. My implementation conforms 100% to the documented behaviour of
the function/type. I should be able to replace __builtin__.file with the
above implementation and expect everything to work (except that I may run
out of memory - but that *also* conforms 100% to documented behaviour).
Indeed, in 2.2.8 File Objects there is in fact no mention that the file
object will be closed when its destructor is called. I searched through much
of the rest of the docs and found no reference to a file object destructor.
Therefore, my above implementation of class file is completely correct in
that it does not have a destructor.

> > You should never rely on automatic freeing of *any* 
> resources unless it is
> > guaranteed. Always explicitly free resources when you have 
> finished with
> > them.
> Well, then you should del all your names too.
> Oops wait, how can we be sure that del works? What if del 
> also increments the 
> refcount instead of decrementing?

"""
__del__(self)

   Called when the instance is about to be destroyed. This is also
   called a destructor  If a base class has a __del__() method, the
   derived class's __del__() method must explicitly call it to ensure
   proper deletion of the base class part of the instance. Note that it
   is possible (though not recommended!) for the __del__() method to
   postpone destruction of the instance by creating a new reference to
   it. It may then be called at a later time when this new reference is
   deleted. It is not guaranteed that __del__() methods are called for
   objects that still exist when the interpreter exits. 

   Note: "del x" doesn't directly call x.__del__() -- the former
   decrements the reference count for x by one, and the latter is only
   called when its reference count reaches zero. Some common situations
   that may prevent the reference count of an object to go to zero
   include: circular references between objects (e.g., a doubly-linked
   list or a tree data structure with parent and child pointers); a
   reference to the object on the stack frame of a function that caught
   an exception (the traceback stored in sys.exc_traceback keeps the
   stack frame alive); or a reference to the object on the stack frame
   that raised an unhandled exception in interactive mode (the traceback
   stored in sys.last_traceback keeps the stack frame alive). The first
   situation can only be remedied by explicitly breaking the cycles; the
   latter two situations can be resolved by storing None in
   sys.exc_traceback or sys.last_traceback. Circular references which
   are garbage are detected when the option cycle detector is enabled
   (it's on by default), but can only be cleaned up if there are no
   Python-level __del__() methods involved. Refer to the documentation
   for the gc module for more information about how __del__() methods
   are handled by the cycle detector, particularly the description of
   the garbage value. 

   Warning: Due to the precarious circumstances under which __del__()
   methods are invoked, exceptions that occur during their execution are
   ignored, and a warning is printed to sys.stderr instead. Also, when
   __del__() is invoked in response to a module being deleted (e.g.,
   when execution of the program is done), other globals referenced by
   the __del__() method may already have been deleted. For this reason,
   __del__() methods should do the absolute minimum needed to maintain
   external invariants. Python 1.5 guarantees that globals whose name
   begins with a single underscore are deleted from their module before
   other globals are deleted; if no other references to such globals
   exist, this may help in assuring that imported modules are still
   available at the time when the __del__() method is called.
"""

An implementation of del which failed to decrement the refcount by one would
not conform to the documented behaviour, and hence would be broken.

Also, in reference to the rest of this thread, note the sentence "It is not
guaranteed that __del__() methods are called for objects that still exist
when the interpreter exits."

> Basically what you have proven is, that if you have a faulty open() 
> implementation, the open(f,t).write(d) idiom doesn't work.

So is the Jython implementation faulty? The open(f, t).write(d) idiom does
not work there (it may, if you get lucky).

> In respect to CPython, refcounting behaviour, or more general 
> retaining 
> references in any gc regime, is a quite important part of the 
> interface.

It is explicitly not a documented part of the interface - the documented
part is that it is not safe to rely on this behaviour.

When it is explicitly documented that this behaviour is guaranteed now and
forever, for all implementations of Python after a specified version, I will
*consider* using this behaviour. Until then it is not worthy of
consideration.

Tim Delaney