Last night I restarted my work on relative redirects in HTTPClientFactory (Ticket #3384). What I had forgotten was how insanely difficult it is to write the test cases, which would then be significantly more bug prone than the code itself. The problem is that for every possible input I need to add (1) A test_* function, (2) a callback, (3) one or more children to the internal web server for the test_* function to connect to.
This morning I hit on another idea: Why not separate the code in HTTPClientFactory that does the fix into a separate function? Then there could be one test case to make sure the code in HTTPClientFactory works, then a single test case that contains a series of assertEquals that test the output of the function. How does that sound? Should I put it outside of HTTPClientFactory or inside?
Cheers! Aaron DeVore
I would suggest rewriting the test cases so you don't need to use an internal webserver. You really just need to be able to send blobs of data to the HTTP client that contain different response codes and Location headers.
I believe there are already some tests in twisted.web.test.test_webclient that do this, such as:
http://twistedmatrix.com/trac/browser/trunk/twisted/web/test/test_webclient.... http://twistedmatrix.com/trac/browser/trunk/twisted/web/test/test_webclient....
Plus there are numerous other examples in twisted of not relying on even the loopback network interface for tests.
-David
On Thu, Mar 5, 2009 at 9:27 AM, Aaron DeVore aaron.devore@gmail.com wrote:
Last night I restarted my work on relative redirects in HTTPClientFactory (Ticket #3384). What I had forgotten was how insanely difficult it is to write the test cases, which would then be significantly more bug prone than the code itself. The problem is that for every possible input I need to add (1) A test_* function, (2) a callback, (3) one or more children to the internal web server for the test_* function to connect to.
This morning I hit on another idea: Why not separate the code in HTTPClientFactory that does the fix into a separate function? Then there could be one test case to make sure the code in HTTPClientFactory works, then a single test case that contains a series of assertEquals that test the output of the function. How does that sound? Should I put it outside of HTTPClientFactory or inside?
Cheers! Aaron DeVore
Twisted-web mailing list Twisted-web@twistedmatrix.com http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-web
On Thu, 5 Mar 2009 10:27:08 -0800, David Reid dreid@dreid.org wrote:
On Thu, Mar 5, 2009 at 9:27 AM, Aaron DeVore aaron.devore@gmail.com wrote:
Last night I restarted my work on relative redirects in HTTPClientFactory (Ticket #3384). What I had forgotten was how insanely difficult it is to write the test cases, which would then be significantly more bug prone than the code itself. The problem is that for every possible input I need to add (1) A test_* function, (2) a callback, (3) one or more children to the internal web server for the test_* function to connect to.
This morning I hit on another idea: Why not separate the code in HTTPClientFactory that does the fix into a separate function? Then there could be one test case to make sure the code in HTTPClientFactory works, then a single test case that contains a series of assertEquals that test the output of the function. How does that sound? Should I put it outside of HTTPClientFactory or inside?
Cheers! Aaron DeVore
I would suggest rewriting the test cases so you don't need to use an internal webserver. You really just need to be able to send blobs of data to the HTTP client that contain different response codes and Location headers.
I believe there are already some tests in twisted.web.test.test_webclient that do this, such as:
http://twistedmatrix.com/trac/browser/trunk/twisted/web/test/test_webclient.... http://twistedmatrix.com/trac/browser/trunk/twisted/web/test/test_webclient....
Plus there are numerous other examples in twisted of not relying on even the loopback network interface for tests.
This is good advice, and it largely agrees with what Aaron was proposing, I think. The core idea here is to only test the code you're trying to test, not a huge heap of other stuff. A lot of the HTTP client tests invoke a huge swath of Twisted Web code, and needlessly so. Finding some way to write new tests which only exercise the very limited set of code that the test is actually for is definitely the way to go.
Jean-Paul
On Thu, Mar 5, 2009 at 10:50 AM, Jean-Paul Calderone exarkun@divmod.com wrote:
This is good advice, and it largely agrees with what Aaron was proposing, I think. The core idea here is to only test the code you're trying to test, not a huge heap of other stuff. A lot of the HTTP client tests invoke a huge swath of Twisted Web code, and needlessly so. Finding some way to write new tests which only exercise the very limited set of code that the test is actually for is definitely the way to go.
Jean-Paul
With a separate function (for brevity I'll just call it r2ar for this email) there would only be a single request to check that r2ar is called at all. The rest of the test would involve calling r2ar a few times. As an added bonus, calls to r2ar are extremely cheap. timeit pegs them at in the order of 10 x -5 seconds on my laptop.
-Aaron
I don't think those would cover what I need. From what I can tell both of those only test the outgoing headers from a request. The fix I'm working on handles an invalid response from the server. The code that I wrote would never even come close to running.
On Thu, Mar 5, 2009 at 10:27 AM, David Reid dreid@dreid.org wrote:
I would suggest rewriting the test cases so you don't need to use an internal webserver. You really just need to be able to send blobs of data to the HTTP client that contain different response codes and Location headers.
I believe there are already some tests in twisted.web.test.test_webclient that do this, such as:
http://twistedmatrix.com/trac/browser/trunk/twisted/web/test/test_webclient.... http://twistedmatrix.com/trac/browser/trunk/twisted/web/test/test_webclient....
Plus there are numerous other examples in twisted of not relying on even the loopback network interface for tests.
-David
On Thu, Mar 5, 2009 at 9:27 AM, Aaron DeVore aaron.devore@gmail.com wrote:
Last night I restarted my work on relative redirects in HTTPClientFactory (Ticket #3384). What I had forgotten was how insanely difficult it is to write the test cases, which would then be significantly more bug prone than the code itself. The problem is that for every possible input I need to add (1) A test_* function, (2) a callback, (3) one or more children to the internal web server for the test_* function to connect to.
This morning I hit on another idea: Why not separate the code in HTTPClientFactory that does the fix into a separate function? Then there could be one test case to make sure the code in HTTPClientFactory works, then a single test case that contains a series of assertEquals that test the output of the function. How does that sound? Should I put it outside of HTTPClientFactory or inside?
Cheers! Aaron DeVore
Twisted-web mailing list Twisted-web@twistedmatrix.com http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-web
Twisted-web mailing list Twisted-web@twistedmatrix.com http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-web
participants (3)
-
Aaron DeVore -
David Reid -
Jean-Paul Calderone