[Python-Dev] Changing all network-oriented tests to facilitate (virtually unlimited) parallel test execution [Issue#: 2550]

[Trent Nelson]

I've forwarding my most recent update to issue 2550 here such that the proposed patch (and in general, the approach to network-oriented test cases) can be vetted by a wider audience:

http://bugs.python.org/file9980/trunk.2550-2.patch

This patch works towards fixing a large proportion of the tests that were written in such a way that often leads to buildbot errors when port conflicts arise.  With this patch applied, I can run many instances of the test suite in parallel and none of the network-oriented tests fail (which they do currently if you try and do this).

Plenty of discussion (mostly me replying to my own comments) on the subject at: http://bugs.python.org/issue2550.

Anyone have any issues with this new approach?  I'm particularily interested in whether or not people disagree with the assumption I've drawn regarding never using SO_REUSEADDR in tests for TCP/IP sockets (see below).

    Trent.


________________________________________
From: python-bugs-list-bounces+tnelson=onresolve.com at python.org [python-bugs-list-bounces+tnelson=onresolve.com at python.org] On Behalf Of Trent Nelson [report at bugs.python.org]
Sent: 08 April 2008 12:49
To: python-bugs-list at python.org
Subject: [issue2550] SO_REUSEADDR doesn't have the same semantics on Windows    as on Unix

Trent Nelson <tnelson at onresolve.com> added the comment:

Invested quite a few cycles on this issue last night.  The more time I
spent on it, the more I became convinced that every single test working
with sockets should be changed in one fell swoop in order to facilitate
(virtually unlimited) parallel test execution without fear of port
conflicts.

I've attached a second patch, trunk.2550-2.patch, which is my progress
so far on doing just this.  The main changes can be expressed by the
following two points:

a) do whatever it takes in network-oriented tests to ensure
   unique ports are obtained (relying on the bind_port() and
   find_unused_port() methods exposed by test_support)

b) never, ever, ever call SO_REUSEADDR on a socket from a test;
   because we're putting so much effort into obtaining a unique
   port, this should never be necessary -- in the rare cases that
   our attempts to obtain a unique port fail, then we absolutely
   should fail with EADDRINUSE, as the ability to obtain a unique
   port for the duration of a client/server test is an invariant
   that we *must* be able to depend upon.  If the invariant is
   broken, fail immediately (don't mask the problem with
   SO_REUSEADDR).

With this patch applied, I can spawn a handful of Python processes and
run the entire test suite (without -r, ensuring all tests are run in
the same order, which should encourage port conflicts (if there were
any)) without any errors*.  Doing that now is completely and utterly
impossible.

[*] Well, almost without error.  All the I/O related tests that try and
open @test fail.

I believe there's still outstanding work to do with this patch with
regards to how the intracacies of SO_REUSEADDR and SO_EXCLUSIVEADDRUSE
should be handled in the rest of the stdlib.  I'm still thinking about
the best approach for this.  However, the patch as it currently stands
is still quite substantial so I wanted to get it out sooner rather than
later for review.

(I'll forward this to python-dev@ to try and encourage more eyes from
people with far more network-fu than I.)

Added file: http://bugs.python.org/file9980/trunk.2550-2.patch

__________________________________
Tracker <report at bugs.python.org>
<http://bugs.python.org/issue2550>
__________________________________
_______________________________________________
Python-bugs-list mailing list
Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/tnelson%40onresolve.com