[Python-Dev] Test failures in test_timeout

[Steve Holden]

Thomas Wouters wrote:
> I'm seeing spurious test failures in test_timeout, on my own workstation and
> on macteagle.python.org (now that it crashes less; Apple sent over some new
> memory.) The problem is pretty simple: both macteagle and my workstation
> live too closely, network-wise, to www.python.org:
> 
> class TimeoutTestCase(unittest.TestCase):
>     [...]
>     def setUp(self):
>         [...]
>         self.addr_remote = ('www.python.org', 80)
>     [...]
>     def testConnectTimeout(self):
>         # Test connect() timeout
>         _timeout = 0.001
>         self.sock.settimeout(_timeout)
> 
>         _t1 = time.time()
>         self.failUnlessRaises(socket.error, self.sock.connect,
>                 self.addr_remote)
> 
> In other words, the test fails because www.python.org responds too quickly.
> 
> The test on my workstation only fails occasionally, but I do expect
> macteagle's failure to be more common (since it's connected to
> www.python.org through (literally) a pair of gigabit switches, whereas my
> workstation has to pass through a few more switches, two Junipers and some
> dark fiber.) Lowering the timeout has no effect, as far as I can tell, which
> is probably a granularity issue.
> 
> I'm thinking that it could probably try to connect to a less reliable
> website, but that's just moving the problem around (and possibly harassing
> an unsuspecting website, particularly around release-time.) Perhaps the test
> should try to connect to a known unconnecting site, like a firewalled port
> on www.python.org? Not something that refuses connections, just something
> that times out.
> 
Couldn't the test use subprocess to start a reliably slow server on 
localhost? It might even be possible to retrieve the ephemeral port 
number used by the server, to avoid conflicts with already-used ports on 
the testing machine.

regards
  Steve
-- 
Steve Holden       +44 150 684 7255  +1 800 494 3119
Holden Web LLC                     www.holdenweb.com
PyCon TX 2006                  www.python.org/pycon/