socket timeouts fail w/ makefile()
I discovered much to my chagrin today that the socket module's new timeout capability doesn't play well with file objects as returned by a socket's makefile method. Tim O'Malley's timeoutsocket module avoids this problem by implementing a simple file-like object directly on top of the socket without calling makefile(). Is there some reason this approach wasn't adopted when adding timeouts to the socket module? I would think the greatest use of timeouts would be using higher-level line-oriented modules like urllib and ftplib. In addition, since makefile() isn't always available, it seems worthwhile to implement something in socket.py, thus making makefile() universally available. I filed a bug report about this issue earlier today in case people are interested: http://www.python.org/sf/707074 Skip
Can you explain better how it doesn't work?
I guess nobody thought of this so far.
Um, when is makefile() not available? There's code for Windows that emulates it, returning a file-like object. Maybe that code should be enabled universally rather than only on Windows...
I'm interested, but have no time... :-( --Guido van Rossum (home page: http://www.python.org/~guido/)
>> I discovered much to my chagrin today that the socket module's new >> timeout capability doesn't play well with file objects as returned by >> a socket's makefile method. Guido> Can you explain better how it doesn't work? When the socket is in non-blocking mode, reads on the file returned by .makefile() will fail with an IOError if there is nothing to return. >> I would think the greatest use of timeouts would be using >> higher-level line-oriented modules like urllib and ftplib. In >> addition, since makefile() isn't always available, it seems >> worthwhile to implement something in socket.py, thus making >> makefile() universally available. Guido> Um, when is makefile() not available? I don't know. I was going by the doc string in socketmodule.c which says, in part: ... makefile([mode, [bufsize]]) -- return a file object for the socket [*]\n\ ... [*] not available on all platforms!"); Maybe the docs are just wrong. According to the #ifdef in the code, if NO_DUP is defined (OS/2, Windows, BeOS), makefile() isn't. Guido> There's code for Windows that emulates it, returning a file-like Guido> object. Maybe that code should be enabled universally rather Guido> than only on Windows... That sounds similar to what is in timeoutsocket.py. It would have the advantage of providing identical semantics across all platforms. Skip
Isn't that exactly what a timeout is supposed to do? What would you have expected?
That's the docs for the _socket module, which is (nowadays) an implementation detail. Read socket.py instead.
Again, I won't have time to do this until after I'm back from Python UK, so I'd appreciate it if someone helped with this, e.g. by filing a patch. --Guido van Rossum (home page: http://www.python.org/~guido/)
>> When the socket is in non-blocking mode, reads on the file returned >> by .makefile() will fail with an IOError if there is nothing to >> return. Guido> Isn't that exactly what a timeout is supposed to do? What would Guido> you have expected? Sorry, I wasn't clear. It fails immediately. The timeout isn't observed. >> makefile([mode, [bufsize]]) -- return a file object for the socket [*]\n\ >> ... >> [*] not available on all platforms!"); Guido> That's the docs for the _socket module, which is (nowadays) an Guido> implementation detail. Read socket.py instead. Maybe _socket shouldn't have such a detailed doc string or should indicate its subservient relationship to socket? I was reading it as if it was a comment in the code, which, in theory, should still be accurate. Guido> Again, I won't have time to do this until after I'm back from Guido> Python UK, so I'd appreciate it if someone helped with this, Guido> e.g. by filing a patch. I'll take a look. There's a bug already in the system (http://www.python.org/sf/707074) to which a patch could be applied, so if someone comes up with something, that's where it goes. Skip
Can you explain better how it doesn't work?
I guess nobody thought of this so far.
Um, when is makefile() not available? There's code for Windows that emulates it, returning a file-like object. Maybe that code should be enabled universally rather than only on Windows...
I'm interested, but have no time... :-( --Guido van Rossum (home page: http://www.python.org/~guido/)
>> I discovered much to my chagrin today that the socket module's new >> timeout capability doesn't play well with file objects as returned by >> a socket's makefile method. Guido> Can you explain better how it doesn't work? When the socket is in non-blocking mode, reads on the file returned by .makefile() will fail with an IOError if there is nothing to return. >> I would think the greatest use of timeouts would be using >> higher-level line-oriented modules like urllib and ftplib. In >> addition, since makefile() isn't always available, it seems >> worthwhile to implement something in socket.py, thus making >> makefile() universally available. Guido> Um, when is makefile() not available? I don't know. I was going by the doc string in socketmodule.c which says, in part: ... makefile([mode, [bufsize]]) -- return a file object for the socket [*]\n\ ... [*] not available on all platforms!"); Maybe the docs are just wrong. According to the #ifdef in the code, if NO_DUP is defined (OS/2, Windows, BeOS), makefile() isn't. Guido> There's code for Windows that emulates it, returning a file-like Guido> object. Maybe that code should be enabled universally rather Guido> than only on Windows... That sounds similar to what is in timeoutsocket.py. It would have the advantage of providing identical semantics across all platforms. Skip
Isn't that exactly what a timeout is supposed to do? What would you have expected?
That's the docs for the _socket module, which is (nowadays) an implementation detail. Read socket.py instead.
Again, I won't have time to do this until after I'm back from Python UK, so I'd appreciate it if someone helped with this, e.g. by filing a patch. --Guido van Rossum (home page: http://www.python.org/~guido/)
>> When the socket is in non-blocking mode, reads on the file returned >> by .makefile() will fail with an IOError if there is nothing to >> return. Guido> Isn't that exactly what a timeout is supposed to do? What would Guido> you have expected? Sorry, I wasn't clear. It fails immediately. The timeout isn't observed. >> makefile([mode, [bufsize]]) -- return a file object for the socket [*]\n\ >> ... >> [*] not available on all platforms!"); Guido> That's the docs for the _socket module, which is (nowadays) an Guido> implementation detail. Read socket.py instead. Maybe _socket shouldn't have such a detailed doc string or should indicate its subservient relationship to socket? I was reading it as if it was a comment in the code, which, in theory, should still be accurate. Guido> Again, I won't have time to do this until after I'm back from Guido> Python UK, so I'd appreciate it if someone helped with this, Guido> e.g. by filing a patch. I'll take a look. There's a bug already in the system (http://www.python.org/sf/707074) to which a patch could be applied, so if someone comes up with something, that's where it goes. Skip
participants (2)
-
Guido van Rossum -
Skip Montanaro