[Python-Dev] test_socketserver failure on cygwin

[Neal Norwitz]

I'm not sure if this is a race condition in the test, the test using
SocketServer inappropriately, or if the failure is exposing a problem
in SocketServer.

The issue is that when the second (UDP) server is setup, there is
still a child process from the first (TCP) server.  So when the UDP
server calls collect_children(), it gets back the pid from the TCP
server, which is not in the UDP server's list and an exception is
raised.  In general, this is a problem, if program is using a
ForkingServer and spawns it's own children, this problem can occur.
This is because os.waitpid() is called with a pid of 0, rather than
the individual pids in the list.  I suppose that would fix the
problem.  Is this worth fixing?  (It's a pain to handle properly
because there is also use of a blocking wait.)

http://www.python.org/dev/buildbot/all/x86%20cygwin%20trunk/builds/1107/step-test/0

Add this patch corrects the problem (see below for more details):

Index: Lib/test/test_socketserver.py
===================================================================
--- Lib/test/test_socketserver.py       (revision 51260)
+++ Lib/test/test_socketserver.py       (working copy)
@@ -183,7 +183,11 @@

 def testall():
     testloop(socket.AF_INET, tcpservers, MyStreamHandler, teststream)
+    time.sleep(.1)
+    reap_children()
     testloop(socket.AF_INET, udpservers, MyDatagramHandler, testdgram)
+    time.sleep(.1)
+    reap_children()
     if hasattr(socket, 'AF_UNIX'):
         testloop(socket.AF_UNIX, streamservers, MyStreamHandler, teststream)
         # Alas, on Linux (at least) recvfrom() doesn't return a meaningful

So modifying SocketServer like this:
-bash-3.1$ svn diff Lib/SocketServer.py Lib/test/test_socketserver.py
Index: Lib/SocketServer.py
===================================================================
--- Lib/SocketServer.py (revision 51260)
+++ Lib/SocketServer.py (working copy)
@@ -420,6 +420,7 @@
             except os.error:
                 pid = None
             if not pid: break
+            print >> sys.__stdout__, 'pid', pid, 'active',
self.active_children             self.active_children.remove(pid)

     def process_request(self, request, client_address):
@@ -428,6 +429,7 @@
         pid = os.fork()
         if pid:
             # Parent process
+            print >> sys.__stdout__, 'in parent, adding', pid,
self.active_children
             if self.active_children is None:
                 self.active_children = []
             self.active_children.append(pid)

We see this on error (prior to the test patch above):

test_socketserver
in parent, adding 3084 None
pid 3084 active [3084]
in parent, adding 2408 []
pid 2408 active [2408]
in parent, adding 3908 []
in parent, adding 532 None
pid 3908 active [532]
Exception in thread Thread-12:
Traceback (most recent call last):
 ...
 File "/home/neal/build/python/trunk/Lib/SocketServer.py", line 424,
in collect_children
    self.active_children.remove(pid)
ValueError: list.remove(x): x not in list