Minor socket timeout quibble - timeout raises socket.error
I just noticed in the development docs that when a timeout on a socket occurs, socket.error is raised. I rather liked the idea that a different exception was raised for timeouts (I used Tim O'Malley's timeout_socket module). Making a TimeoutError exception a subclass of socket.error would be fine so you can catch it with existing code, but I could see recovering differently for a timeout as opposed to other possible errors: sock.settimeout(5.0) try: data = sock.recv(8192) except socket.TimeoutError: # maybe requeue the request ... except socket.error, codes: # some more drastic solution is needed ... Skip
Skip Montanaro wrote:
I just noticed in the development docs that when a timeout on a socket occurs, socket.error is raised. I rather liked the idea that a different exception was raised for timeouts (I used Tim O'Malley's timeout_socket module). Making a TimeoutError exception a subclass of socket.error would be fine so you can catch it with existing code, but I could see recovering differently for a timeout as opposed to other possible errors:
sock.settimeout(5.0) try: data = sock.recv(8192) except socket.TimeoutError: # maybe requeue the request ... except socket.error, codes: # some more drastic solution is needed ...
+1 on your suggestion. Anyway, under windows, the current implementation returns incorrect socket.error code for timeout. I am working on the test suite as well as a fix for problem found. Once the code is bug free maybe we can put the TimeoutError in. I will leave it to Guido for the approval of the change. When he comes back from his holiday. Bernie
Skip
_______________________________________________ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev
----- Original Message -----
From: "Skip Montanaro"
I just noticed in the development docs that when a timeout on a socket occurs, socket.error is raised. I rather liked the idea that a different exception was raised for timeouts (I used Tim O'Malley's timeout_socket module). Making a TimeoutError exception a subclass of socket.error would be fine so you can catch it with existing code, but I could see recovering differently for a timeout as opposed to other possible errors:
sock.settimeout(5.0) try: data = sock.recv(8192) except socket.TimeoutError: # maybe requeue the request ... except socket.error, codes: # some more drastic solution is needed ...
This seems logical: the timeout is inherently different, so a separate "except" seems better than having to analyze the reason of the socket error. regards ----------------------------------------------------------------------- Steve Holden http://www.holdenweb.com/ Python Web Programming http://pydish.holdenweb.com/pwp/ -----------------------------------------------------------------------
[Skip Montanaro]
I just noticed in the development docs that when a timeout on a socket occurs, socket.error is raised. I rather liked the idea that a different exception was raised for timeouts (I used Tim O'Malley's timeout_socket module). Making a TimeoutError exception a subclass of socket.error would be fine so you can catch it with existing code, but I could see recovering differently for a timeout as opposed to other possible errors:
sock.settimeout(5.0) try: data = sock.recv(8192) except socket.TimeoutError: # maybe requeue the request ... except socket.error, codes: # some more drastic solution is needed ...
[Bernard Yue]
+1 on your suggestion. Anyway, under windows, the current implementation returns incorrect socket.error code for timeout. I am working on the test suite as well as a fix for problem found. Once the code is bug free maybe we can put the TimeoutError in.
I will leave it to Guido for the approval of the change. When he comes back from his holiday.
The way I restructured the code it is impossible to distinguish a timeout error from other errors; you simply get the "no data available" error from the socket operation. This is the same error you'd get in non-blocking mode. Before I recomplicate the code so that it can raise a separate error when the select fails, I'd like to understand the use case better. Why would you want to make this distinction? Requeueing the request (as in Skip's example) doesn't make sense IMO: you set the timeout for a reason, and that reason is that you want to give up if it takes too long. If you really intend to retry you're better of disabling the timeout! If you really want to, you can already distinguish the timeout case, because you get an EAGAIN error then (maybe something else on Windows -- Bernard, if you have a fix for that, please send it to me). So a -0 unless more evidence is brought forward. --Guido van Rossum (home page: http://www.python.org/~guido/)
Guido van Rossum wrote:
[Skip Montanaro]
I just noticed in the development docs that when a timeout on a socket occurs, socket.error is raised. I rather liked the idea that a different exception was raised for timeouts (I used Tim O'Malley's timeout_socket module). Making a TimeoutError exception a subclass of socket.error would be fine so you can catch it with existing code, but I could see recovering differently for a timeout as opposed to other possible errors:
sock.settimeout(5.0) try: data = sock.recv(8192) except socket.TimeoutError: # maybe requeue the request ... except socket.error, codes: # some more drastic solution is needed ...
[Bernard Yue]
+1 on your suggestion. Anyway, under windows, the current implementation returns incorrect socket.error code for timeout. I am working on the test suite as well as a fix for problem found. Once the code is bug free maybe we can put the TimeoutError in.
I will leave it to Guido for the approval of the change. When he comes back from his holiday.
The way I restructured the code it is impossible to distinguish a timeout error from other errors; you simply get the "no data available" error from the socket operation. This is the same error you'd get in non-blocking mode.
To distinguish a timeout error, the caller can check s->sock_timeout when a non-blocking mode error occured, or just return an error code from internal_select() (I guess you must have your reason to taken it out in the first place)
Before I recomplicate the code so that it can raise a separate error when the select fails, I'd like to understand the use case better. Why would you want to make this distinction? Requeueing the request (as in Skip's example) doesn't make sense IMO: you set the timeout for a reason, and that reason is that you want to give up if it takes too long. If you really intend to retry you're better of disabling the timeout!
How about the following (assume we have socket.setDefaultTimeout()): import socket import urllib socket.setDefaultTimeout(5.0) retry = 0 url = 'some url' while retry < 3: try: file = urllib.urlretrieve(url) except socket.TimeoutError: if retry == 2: print "Server too busy, given up!" raise else: print "Server busy, retry!" retry += 1 else: break MS IIS behave strangely to http request. When the server is very busy, it will randomly drop some requests without disconnecting the client. So the best approach for the client is to timeout and retry. I guess that might be the reason why people needed timeoutsocket in the first place.
If you really want to, you can already distinguish the timeout case, because you get an EAGAIN error then (maybe something else on Windows -- Bernard, if you have a fix for that, please send it to me).
I am struggling with the test case for the new socket code. The timeout test case I've send you works with the old socketmodule.c (attached), but not with the lastest version (on linux or windows). It's strange, your new implementation looks much cleaner. Please bear with me a bit longer for a patch :.(
So a -0 unless more evidence is brought forward.
--Guido van Rossum (home page: http://www.python.org/~guido/)
Bernie
The way I restructured the code it is impossible to distinguish a timeout error from other errors; you simply get the "no data available" error from the socket operation. This is the same error you'd get in non-blocking mode.
To distinguish a timeout error, the caller can check s->sock_timeout when a non-blocking mode error occured, or just return an error code from internal_select() (I guess you must have your reason to taken it out in the first place)
I don't understand your first suggestion. Not all errors mean that the timeout triggered! I took it out because it is much less code this way.
Before I recomplicate the code so that it can raise a separate error when the select fails, I'd like to understand the use case better. Why would you want to make this distinction? Requeueing the request (as in Skip's example) doesn't make sense IMO: you set the timeout for a reason, and that reason is that you want to give up if it takes too long. If you really intend to retry you're better of disabling the timeout!
How about the following (assume we have socket.setDefaultTimeout()):
import socket import urllib
socket.setDefaultTimeout(5.0) retry = 0 url = 'some url'
while retry < 3: try: file = urllib.urlretrieve(url) except socket.TimeoutError: if retry == 2: print "Server too busy, given up!" raise else: print "Server busy, retry!" retry += 1 else: break
MS IIS behave strangely to http request. When the server is very busy, it will randomly drop some requests without disconnecting the client. So the best approach for the client is to timeout and retry. I guess that might be the reason why people needed timeoutsocket in the first place.
One of the reasons (there are lots of reasons why a connect or receive attempt may be very slow to time out, or even never time out). Of course, this stll doesn't distinguish between a timeout from connect() and one from recv(). Have you ever written code like this?
If you really want to, you can already distinguish the timeout case, because you get an EAGAIN error then (maybe something else on Windows -- Bernard, if you have a fix for that, please send it to me).
I am struggling with the test case for the new socket code. The timeout test case I've send you works with the old socketmodule.c (attached), but not with the lastest version (on linux or windows). It's strange, your new implementation looks much cleaner.
No need to attach copies of old versions -- just give me the CVS revision number. :-)
Please bear with me a bit longer for a patch :.(
OK. Anyway, I have no time to play with this right now, so I'm glad you aren't giving up just yet. :-) --Guido van Rossum (home page: http://www.python.org/~guido/)
Guido van Rossum wrote:
To distinguish a timeout error, the caller can check s->sock_timeout when a non-blocking mode error occured, or just return an error code from internal_select() (I guess you must have your reason to taken it out in the first place)
I don't understand your first suggestion. Not all errors mean that the timeout triggered!
For example, when accept() fail with error code EAGAIN and s->sock_timeout = 5.0, it indicates timeout. Same for connect() fail with EINPROGRESS. Anyway, on second thought, it is messy.
How about the following (assume we have socket.setDefaultTimeout()):
import socket import urllib
socket.setDefaultTimeout(5.0) retry = 0 url = 'some url'
while retry < 3: try: file = urllib.urlretrieve(url) except socket.TimeoutError: if retry == 2: print "Server too busy, given up!" raise else: print "Server busy, retry!" retry += 1 else: break
MS IIS behave strangely to http request. When the server is very busy, it will randomly drop some requests without disconnecting the client. So the best approach for the client is to timeout and retry. I guess that might be the reason why people needed timeoutsocket in the first place.
One of the reasons (there are lots of reasons why a connect or receive attempt may be very slow to time out, or even never time out).
Of course, this stll doesn't distinguish between a timeout from connect() and one from recv().
I think you are right on the point. Client might not care if the call is timeouted on connect() or recv(). In this case a timeout error comes handy.
Have you ever written code like this?
Yes I did.
I am struggling with the test case for the new socket code. The timeout test case I've send you works with the old socketmodule.c (attached), but not with the lastest version (on linux or windows). It's strange, your new implementation looks much cleaner.
No need to attach copies of old versions -- just give me the CVS revision number. :-)
socketmodule.c version 1.225 socketmodule.h version 1.7
Please bear with me a bit longer for a patch :.(
OK.
Anyway, I have no time to play with this right now, so I'm glad you aren't giving up just yet. :-)
It is very painful indeed (Tim was so right).
--Guido van Rossum (home page: http://www.python.org/~guido/)
Bernie
>> > I just noticed in the development docs that when a timeout on a >> > socket occurs, socket.error is raised. I rather liked the idea >> > that a different exception was raised for timeouts (I used Tim >> > O'Malley's timeout_socket module). Guido> I'd like to understand the use case better. Why would you want Guido> to make this distinction? In my application that uses Tim O'Malley's timeout_socket module, I do very little different in the two cases other than to generate a different message for the user. As I mentioned in the subject, it is a minor quibble. If it's a pain to modify the code to raise a distinct error, I wouldn't bother. Skip
participants (4)
-
Bernard Yue -
Guido van Rossum -
Skip Montanaro -
Steve Holden