close(), exceptions and problems

[Antti Kuntsi]

Hi,
I have run into the following problem with python and close()s. When the
system is under a heavy load the close() system call can be interrupted,
but the Python never detects this. This leads into a lot of problems,
including at least the following:

socket object: the socket is not closed, and the FD reserved for the
connection is left to the process, but the FD-number is lost.

file object: the file is not closed, FD is left unreferenced.

the pipes from popen2.popen3(): The pipe(s) are left open, eg a sub-process
waiting for EOF waits forever.

So far I have checked the implementation of the socket object .close() -method
and it ignores the return value from close().

As a result Python becomes extremely unstable under if the system is under
a heavy load. Circumventing this at python-level does not work, as the
following code can go thru without executing the except-branch in a
multithreaded application regardless of if the close fails or succeeds.
Here's a quick piece of code demonstrating how not to avoid the problem,
even with stricter except:s you cannot guarantee that the socket has been
closed.

import os
...
def badClose(socket_object):
	fd=socket_object.fileno() # we loose the FD number in close()
	stats=os.fstat(fd)
	# Today we don't know if this succeeds or not
	socket_object.close() 
	try:
		if os.fstat(fd) == stats:
			raise Exception, "The socket was not closed"
# or so you'd think. the fstat() results can be identical, at least on
# Solaris 7. The socket could have been closed and a new socket with the
# same ctime and inode might have already been given out between the
# close() and the seconds fstat()
	except: # fstat() throws an exception if the FD is bad
		print "Socket was closed"

This seems to affect at least Python 1.5.2 and Python 2.0.


Antti Kuntsi

-- 
         /"\                           |    iki.
         \ /     ASCII Ribbon Campaign |    fi/
          X      Against HTML Mail     |    mickut
         / \