[Python-3000] socket GC worries

Bill Janssen janssen at parc.com
Sun Oct 28 20:09:56 CET 2007


I've now got a working SSL patch for Py3K (assuming that the patches
for #1347 and #1349 are committed), but I'm a bit worried about the
lazy GC of sockets.  I find that simply dropping an SSLSocket on the
floor doesn't GC the C structures.  This implies that the instance in
the SSLSocket._sslobj slot never gets decref'ed.  I think it's due to
the fact that "socket.makefile()" creates a circular reference with an
instance of "socket.SocketCloser", which points to the socket, and the
socket has a slot which points to the "_closer".  If "socket.close()"
is never explicitly called, the underlying system socket never gets
closed.  Since sockets are bound to a scarce system resource, this
could be problematic.

I think that the SocketCloser (new in Py3K) was developed to address
another issue, which is that there's a lot of library code which
assumes that the Python socket instance is just window dressing over
an underlying system file descriptor, and isn't important.  In fact,
that whole mess of code is a good argument for *not* exposing the
fileno in Python (perhaps only for special cases, like "select").
Take httplib and urllib, for instance.  HTTPConnection creates a
"file" from the socket, by calling socket.makefile(), then in some
cases *closes* the socket (thereby reasonably rendering the socket
*dead*), *then* returns the "file" to the caller as part of the
response.  urllib then takes the response, pulls the "file" out of it,
and discards the rest, returning the "file" as part of an instance of
addinfourl.  Somewhere along the way some code should call "close()"
on that HTTPConnection socket, but not till the caller is finished
using the bytes of the response (and those bytes are kept queued up in
the real OS socket).  Ideally, GC of the response instance should call
close() on the socket instance, which means that the instance should
be passed along as part of the response, IMO.

Bill


More information about the Python-3000 mailing list