On Wed, Mar 20, 2013 at 12:47:09PM -0700, Noah Kantrowitz wrote:

On Mar 20, 2013, at 12:44 PM, Martin v. Löwis wrote:

...

Am 20.03.13 04:27, schrieb R. David Murray:

...

Example:

request = urllib.request.Request("http://www.python.org/~jeremy/")

And then there are things like:

h = client.HTTPSConnection('svn.python.org', 443, context=context)

I don't think there are a huge number of these, but we will need to deal with them.

In the past, we just let them break and then fix the test case, which is not the best solution, but also not overly tragic (IMO).

Note that the redesign project includes an objective of URL stability. The test suite is here of least concern, as there are thousands of incoming links to various pages. These need to be preserved as much as feasible (possibly using redirects).

+1 on let them break, but when they get fixed it should be against a dedicated testing endpoint if possible, preferably something internal to the test suite. Maybe spawn an HTTP server in a background thread/proc? I would very much like the web infra to not be a runtime dependency if possible.

FWIW, that approach isn't possible in all cases.  Perfect example
was http://bugs.python.org/issue15285, which required very specific
TCP/IP behavior (specifically with regards to RST) that you simply
can't simulate easily on a localhost.

I set up two services, blackhole and whitehole.snakebite.net, that
have been fulfilling this need for the past ~10 months now.

Relevant blurb below:

def testConnectTimeout(self):
    # Testing connect timeout is tricky: we need to have IP connectivity
    # to a host that silently drops our packets.  We can't simulate this
    # from Python because it's a function of the underlying TCP/IP stack.
    # So, the following Snakebite host has been defined:
    blackhole = ('blackhole.snakebite.net', 56666)

    # Blackhole has been configured to silently drop any incoming packets.
    # No RSTs (for TCP) or ICMP UNREACH (for UDP/ICMP) will be sent back
    # to hosts that attempt to connect to this address: which is exactly
    # what we need to confidently test connect timeout.

    # However, we want to prevent false positives.  It's not unreasonable
    # to expect certain hosts may not be able to reach the blackhole, due
    # to firewalling or general network configuration.  In order to improve
    # our confidence in testing the blackhole, a corresponding 'whitehole'
    # has also been set up using one port higher:
    whitehole = ('whitehole.snakebite.net', 56667)

    # This address has been configured to immediately drop any incoming
    # packets as well, but it does it respectfully with regards to the
    # incoming protocol.  RSTs are sent for TCP packets, and ICMP UNREACH
    # is sent for UDP/ICMP packets.  This means our attempts to connect to
    # it should be met immediately with ECONNREFUSED.  The test case has
    # been structured around this premise: if we get an ECONNREFUSED from
    # the whitehole, we proceed with testing connect timeout against the
    # blackhole.  If we don't, we skip the test (with a message about not
    # getting the required RST from the whitehole within the required
    # timeframe).

    # For the records, the whitehole/blackhole configuration has been set
    # up using the 'pf' firewall (available on BSDs), using the following:
    #
    #   ext_if="bge0"
    #
    #   blackhole_ip="35.8.247.6"
    #   whitehole_ip="35.8.247.6"
    #   blackhole_port="56666"
    #   whitehole_port="56667"
    #
    #   block return in log quick on $ext_if proto { tcp udp } \
    #       from any to $whitehole_ip port $whitehole_port
    #   block drop in log quick on $ext_if proto { tcp udp } \
    #       from any to $blackhole_ip port $blackhole_port
    #

Regards,

    Trent.