[Python-Dev] Pervasive socket failures on Windows

[Tim Peters]

Noticed that various socket tests are failing today, WinXP, Python trunk:

test_socket_ssl
Exception in thread Thread-27:
Traceback (most recent call last):
  File "C:\Code\python\lib\threading.py", line 444, in __bootstrap
    self.run()
  File "C:\Code\python\lib\threading.py", line 424, in run
    self.__target(*self.__args, **self.__kwargs)
  File "C:\Code\python\lib\test\test_socket_ssl.py", line 50, in listener
    s.accept()
  File "C:\Code\python\lib\socket.py", line 169, in accept
    sock, addr = self._sock.accept()
error: unable to select on socket

test test_socket_ssl crashed -- socket.error: (10061, 'Connection refused')



test test_urllibnet failed -- errors occurred; run in verbose mode for details
Running that in verbose mode shows 2 "ok" and 8 "ERROR".  A typical ERROR:

ERROR: test_basic (test.test_urllibnet.urlopenNetworkTests)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "C:\Code\python\lib\test\test_urllibnet.py", line 43, in test_basic
    open_url = urllib.urlopen("http://www.python.org/")
  File "C:\Code\python\lib\urllib.py", line 82, in urlopen
    return opener.open(url)
  File "C:\Code\python\lib\urllib.py", line 190, in open
    return getattr(self, name)(url)
  File "C:\Code\python\lib\urllib.py", line 325, in open_http
    h.endheaders()
  File "C:\Code\python\lib\httplib.py", line 798, in endheaders
    self._send_output()
  File "C:\Code\python\lib\httplib.py", line 679, in _send_output
    self.send(msg)
  File "C:\Code\python\lib\httplib.py", line 658, in send
    self.sock.sendall(str)
  File "<string>", line 1, in sendall
IOError: [Errno socket error] unable to select on socket



test_logging appears to consume 100% of a CPU now, and never finishes.
 This may be an independent error.


test_asynchat
Exception in thread Thread-1:
Traceback (most recent call last):
  File "C:\Code\python\lib\threading.py", line 444, in __bootstrap
    self.run()
  File "C:\Code\python\lib\test\test_asynchat.py", line 18, in run
    conn, client = sock.accept()
  File "C:\Code\python\lib\socket.py", line 169, in accept
    sock, addr = self._sock.accept()
error: unable to select on socket


test_socket is a long-winded disaster.


test_socketserver
test test_socketserver crashed -- socket.error: (10061, 'Connection refused')


There are others, but tests that use sockets hang a lot now & it's
tedious to worm around that.


I _suspect_ that rev 42253 introduced these problems.  For example, that added:

+       /* Guard against socket too large for select*/
+       if (s->sock_fd >= FD_SETSIZE)
+               return SOCKET_INVALID;

to _ssl.c, and added

+/* Can we call select() with this socket without a buffer overrun? */
+#define IS_SELECTABLE(s) ((s)->sock_fd < FD_SETSIZE)

to socketmodule.c, but those appear to make no sense.  FD_SETSIZE is
the maximum number of distinct fd's an fdset can hold, and the
numerical magnitude of any specific fd has nothing to do with that in
general (they may be related in fact on Unix systems that implement an
fdset as "a big bit vector" -- but Windows doesn't work that way, and
neither do all Unix systems, and nothing in socket specs requires an
implementation to work that way).