Exceptions and Object Destruction (was: Problem with apsw and garbage collection)
steven at REMOVE.THIS.cybersource.com.au
Wed Jun 17 23:41:55 EDT 2009
On Wed, 17 Jun 2009 22:58:27 -0400, Charles Yeomans wrote:
> On Jun 17, 2009, at 9:43 PM, Steven D'Aprano wrote:
>> On Wed, 17 Jun 2009 07:49:52 -0400, Charles Yeomans wrote:
>>>> Even CPython doesn't rely completely on reference counting (it has a
>>>> fallback gc for cyclic garbage). Python introduced the "with"
>>>> statement to get away from the kludgy CPython programmer practice of
>>>> opening files and relying on the file being closed when the last
>>>> reference went out of scope.
>>> I'm curious as you why you consider this practice to be kludgy; my
>>> experience with RAII is pretty good.
>> Because it encourages harmful laziness. Laziness is only a virtue when
>> leads to good code for little effort, but in this case, it leads to
>> portable code. Worse, if your data structures include cycles, it also
>> leads to resource leaks.
> Memory management may be an "implementation detail", but it is
> unfortunately one that illustrates the so-called law of leaky
> abstractions. So I think that one has to write code that follows the
> memory management scheme of whatever language one uses. For code
> written for CPython only, as mine is, RAII is an appropriate idiom and
> not kludgy at all. Under your assumptions, its use would be wrong, of
CPython isn't a language, it's an implementation.
I'm unable to find anything in the Python Reference which explicitly
states that files will be closed when garbage collected, except for one
brief mention in tempfile.TemporaryFile:
"Return a file-like object that can be used as a temporary storage area.
The file is created using mkstemp(). It will be destroyed as soon as it
is closed (including an implicit close when the object is garbage
In practical terms, it's reasonably safe to assume Python will close
files when garbage collected (it would be crazy not to!) but that's not
explicitly guaranteed anywhere I can see. In any case, there is no
guarantee *when* files will be closed -- for long-lasting processes that
open and close a lot of files, or for data structures with cycles, you
might easily run out of file descriptors.
The docs for file give two recipes for recommended ways of dealing with
Both of them close the file, one explicitly, the other implicitly. In
both cases, they promise to close the file as soon as you are done with
it. Python the language does not.
The tutorials explicitly recommends closing the file when you're done:
"When you’re done with a file, call f.close() to close it and free up any
system resources taken up by the open file."
In summary: relying on immediate closure of files is implementation
specific behaviour. By all means do so, with your eyes open and with full
understanding that you're relying on platform-specific behaviour with no
guarantee of when, or even if, files will be closed.
More information about the Python-list