socket timeouts fail w/ makefile()
I discovered much to my chagrin today that the socket module's new timeout capability doesn't play well with file objects as returned by a socket's makefile method. Tim O'Malley's timeoutsocket module avoids this problem by implementing a simple file-like object directly on top of the socket without calling makefile(). Is there some reason this approach wasn't adopted when adding timeouts to the socket module? I would think the greatest use of timeouts would be using higher-level line-oriented modules like urllib and ftplib. In addition, since makefile() isn't always available, it seems worthwhile to implement something in socket.py, thus making makefile() universally available. I filed a bug report about this issue earlier today in case people are interested: http://www.python.org/sf/707074 Skip
I discovered much to my chagrin today that the socket module's new timeout capability doesn't play well with file objects as returned by a socket's makefile method.
Can you explain better how it doesn't work?
Tim O'Malley's timeoutsocket module avoids this problem by implementing a simple file-like object directly on top of the socket without calling makefile(). Is there some reason this approach wasn't adopted when adding timeouts to the socket module?
I guess nobody thought of this so far.
I would think the greatest use of timeouts would be using higher-level line-oriented modules like urllib and ftplib. In addition, since makefile() isn't always available, it seems worthwhile to implement something in socket.py, thus making makefile() universally available.
Um, when is makefile() not available? There's code for Windows that emulates it, returning a file-like object. Maybe that code should be enabled universally rather than only on Windows...
I filed a bug report about this issue earlier today in case people are interested:
I'm interested, but have no time... :-( --Guido van Rossum (home page: http://www.python.org/~guido/)
>> I discovered much to my chagrin today that the socket module's new >> timeout capability doesn't play well with file objects as returned by >> a socket's makefile method. Guido> Can you explain better how it doesn't work? When the socket is in non-blocking mode, reads on the file returned by .makefile() will fail with an IOError if there is nothing to return. >> I would think the greatest use of timeouts would be using >> higher-level line-oriented modules like urllib and ftplib. In >> addition, since makefile() isn't always available, it seems >> worthwhile to implement something in socket.py, thus making >> makefile() universally available. Guido> Um, when is makefile() not available? I don't know. I was going by the doc string in socketmodule.c which says, in part: ... makefile([mode, [bufsize]]) -- return a file object for the socket [*]\n\ ... [*] not available on all platforms!"); Maybe the docs are just wrong. According to the #ifdef in the code, if NO_DUP is defined (OS/2, Windows, BeOS), makefile() isn't. Guido> There's code for Windows that emulates it, returning a file-like Guido> object. Maybe that code should be enabled universally rather Guido> than only on Windows... That sounds similar to what is in timeoutsocket.py. It would have the advantage of providing identical semantics across all platforms. Skip
>> I discovered much to my chagrin today that the socket >> module's new timeout capability doesn't play well with file >> objects as returned by a socket's makefile method.
Guido> Can you explain better how it doesn't work?
When the socket is in non-blocking mode, reads on the file returned by .makefile() will fail with an IOError if there is nothing to return.
Isn't that exactly what a timeout is supposed to do? What would you have expected?
>> I would think the greatest use of timeouts would be using >> higher-level line-oriented modules like urllib and ftplib. In >> addition, since makefile() isn't always available, it seems >> worthwhile to implement something in socket.py, thus making >> makefile() universally available.
Guido> Um, when is makefile() not available?
I don't know. I was going by the doc string in socketmodule.c which says, in part:
... makefile([mode, [bufsize]]) -- return a file object for the socket [*]\n\ ... [*] not available on all platforms!");
Maybe the docs are just wrong. According to the #ifdef in the code, if NO_DUP is defined (OS/2, Windows, BeOS), makefile() isn't.
That's the docs for the _socket module, which is (nowadays) an implementation detail. Read socket.py instead.
Guido> There's code for Windows that emulates it, returning a Guido> file-like object. Maybe that code should be enabled Guido> universally rather than only on Windows...
That sounds similar to what is in timeoutsocket.py. It would have the advantage of providing identical semantics across all platforms.
Again, I won't have time to do this until after I'm back from Python UK, so I'd appreciate it if someone helped with this, e.g. by filing a patch. --Guido van Rossum (home page: http://www.python.org/~guido/)
>> When the socket is in non-blocking mode, reads on the file returned >> by .makefile() will fail with an IOError if there is nothing to >> return. Guido> Isn't that exactly what a timeout is supposed to do? What would Guido> you have expected? Sorry, I wasn't clear. It fails immediately. The timeout isn't observed. >> makefile([mode, [bufsize]]) -- return a file object for the socket [*]\n\ >> ... >> [*] not available on all platforms!"); Guido> That's the docs for the _socket module, which is (nowadays) an Guido> implementation detail. Read socket.py instead. Maybe _socket shouldn't have such a detailed doc string or should indicate its subservient relationship to socket? I was reading it as if it was a comment in the code, which, in theory, should still be accurate. Guido> Again, I won't have time to do this until after I'm back from Guido> Python UK, so I'd appreciate it if someone helped with this, Guido> e.g. by filing a patch. I'll take a look. There's a bug already in the system (http://www.python.org/sf/707074) to which a patch could be applied, so if someone comes up with something, that's where it goes. Skip
participants (2)
-
Guido van Rossum -
Skip Montanaro