[Python-3000] socket GC worries

[Guido van Rossum]

2007/10/29, Greg Ewing <greg.ewing at canterbury.ac.nz>:
> I wrote:
>
>  > Seems to me that a socket should already *be* a file,
>  > so it shouldn't need a makefile() method and you
>  > shouldn't have to mess around with filenos.
>
> Guido van Rossum wrote:
>
> > That model fits TCP/IP streams just fine, but doesn't work so well for
> > UDP and other odd socket types.
>
> No, but I think that a socket should have read() and
> write() methods that work if it happens to be a socket
> of an appropriate kind. Unix lets you use read and write
> as synonyms for send and recv on stream sockets, and
> it's surprising that Python doesn't do the same.

That's because I don't find the synonyms a good idea.

> At the very least, it should be possible to wrap
> any of the higher-level I/O stack objects around a
> stream socket directly.

Why? What problem does this solve?

> > The real issue seems to be file descriptor GC. Maybe we haven't
> > written down the rules clearly enough for when the fd is supposed to
> > be GC'ed
>
> I don't see what's so difficult about this. Each file
> descriptor should be owned by exactly one object. If
> two objects need to share a fd, then you dup() it so
> that each one has its own fd. When the object is
> close()d or GCed, it closes its fd.

On Windows you can't dup() a fd.

> However, I don't see that it should be necessary for
> objects to share fds in the first place. Buffering
> layers should wrap directly around the the object
> being buffered, whether a file or socket or something
> else. Then whether the socket has a fd or not is
> an implementation detail of the socket object, so
> there's no problem on Windows.

There's a tension though between using GC and explicit closing. A
fairly nice model would be that the lowest-level object "owns" the fd
and is the one to close it when it is GC'ed. However for various
reasons we don't want to rely on GC to close fds, since that may delay
closing in Jython and when there happens to be an innocent reference
keeping the lowest-level socket object alive (e.g. someone still has
it in their stack frame or traceback). So we end up having to
implement a second reference counting scheme on top of close() calls.
Which is what we did. But now just dropping the last reference to an
object doesn't call close(), so explicit closes suddenly become
mandatory instead of recommended good practice. Adding __del__ as an
alias for close might help, except this makes circular references a
primary sin (since the cycle GC doesn't like calling __del__). I guess
there really is no way around this solution though, and we'll just
have to make extra sure not to create cycles during normal usage
patterns, or use weak references in those cases where we can't avoid
them.

I think this is the way to go, together with changing the Socket class
from subclassing _socket to wrapping one.

--Guido

> Bill Janssen wrote:
>
> > Back to your initial mail (which is
> > more relevant than Greg Ewing's snipe!):
>
> What snipe? I'm trying to make a constructive suggestion.
>
> > then in some
> > cases *closes* the socket (thereby reasonably rendering the socket
> > *dead*), *then* returns the "file" to the caller as part of the
> > response.
>
> I don't understand that. What good can returning a *closed* file
> object possibly do anyone?
>
> --
> Greg
> _______________________________________________
> Python-3000 mailing list
> Python-3000 at python.org
> http://mail.python.org/mailman/listinfo/python-3000
> Unsubscribe: http://mail.python.org/mailman/options/python-3000/guido%40python.org
>


-- 
--Guido van Rossum (home page: http://www.python.org/~guido/)