Exceptions and Object Destruction (was: Problem with apsw and garbage collection)

Steven D'Aprano steven at REMOVE.THIS.cybersource.com.au
Wed Jun 17 23:41:55 EDT 2009


On Wed, 17 Jun 2009 22:58:27 -0400, Charles Yeomans wrote:

> On Jun 17, 2009, at 9:43 PM, Steven D'Aprano wrote:
> 
>> On Wed, 17 Jun 2009 07:49:52 -0400, Charles Yeomans wrote:
>>
>>>> Even CPython doesn't rely completely on reference counting (it has a
>>>> fallback gc for cyclic garbage).  Python introduced the "with"
>>>> statement to get away from the kludgy CPython programmer practice of
>>>> opening files and relying on the file being closed when the last
>>>> reference went out of scope.
>>>
>>> I'm curious as you why you consider this practice to be kludgy; my
>>> experience with RAII is pretty good.
>>
>> Because it encourages harmful laziness. Laziness is only a virtue when
>> it
>> leads to good code for little effort, but in this case, it leads to
>> non-
>> portable code. Worse, if your data structures include cycles, it also
>> leads to resource leaks.
>>
>>
> 
> Memory management may be an "implementation detail", but it is
> unfortunately one that illustrates the so-called law of leaky
> abstractions.  So I think that one has to write code that follows the
> memory management scheme of whatever language one uses.   For code
> written for CPython only, as mine is, RAII is an appropriate idiom and
> not kludgy at all.  Under your assumptions, its use would be wrong, of
> course.


CPython isn't a language, it's an implementation.

I'm unable to find anything in the Python Reference which explicitly 
states that files will be closed when garbage collected, except for one 
brief mention in tempfile.TemporaryFile:

"Return a file-like object that can be used as a temporary storage area. 
The file is created using mkstemp(). It will be destroyed as soon as it 
is closed (including an implicit close when the object is garbage 
collected)."

http://docs.python.org/library/tempfile.html

In practical terms, it's reasonably safe to assume Python will close 
files when garbage collected (it would be crazy not to!) but that's not 
explicitly guaranteed anywhere I can see. In any case, there is no 
guarantee *when* files will be closed -- for long-lasting processes that 
open and close a lot of files, or for data structures with cycles, you 
might easily run out of file descriptors.


The docs for file give two recipes for recommended ways of dealing with 
files:

http://docs.python.org/library/stdtypes.html#file.close

Both of them close the file, one explicitly, the other implicitly. In 
both cases, they promise to close the file as soon as you are done with 
it. Python the language does not.


The tutorials explicitly recommends closing the file when you're done:

"When you’re done with a file, call f.close() to close it and free up any 
system resources taken up by the open file."

http://docs.python.org/tutorial/inputoutput.html#reading-and-writing-files


In summary: relying on immediate closure of files is implementation 
specific behaviour. By all means do so, with your eyes open and with full 
understanding that you're relying on platform-specific behaviour with no 
guarantee of when, or even if, files will be closed.



-- 
Steven



More information about the Python-list mailing list