[Python-Dev] httplib and bad response chunking
John J Lee
jjl at pobox.com
Wed Aug 9 22:40:25 CEST 2006
On Sun, 30 Jul 2006, Greg Ward wrote:
> Did you look at the crude attempt at testing for this bug that I hacked
> into test_httplib.py? I posted it to bug #1486335 here:
> The idea is simple: put various chunked responses into strings and then
> feed those strings to HTTPConnection. The trouble is that StringIO does
> not behave the same as a real socket: where HTTPResponse fails one way
> reading from a real socket (eg. infinite loop), it fails differently (or
> not at all) reading from a StringIO. Makes testing with the FakeSocket
> class in test_httplib.py problematic.
There are always ways around unit testing problems, but...
> Maybe the right way to test httplib is to spawn a server process
> (thread?) to listen on some random port, feed various HTTP responses at
> HTTPConnection/HTTPResponse, and see what happens. I'm not sure how to
> do that portably, though. Well, I'll see if I can whip up a Unix-y
> solution and see if anyone knows how to make it portable.
I think adding this kind of functional test is extremely valuable, given
that we don't seem to have any for httplib at present
(Lib/test/test_httplib.py does not send any network packets). Maybe you
could add a basic smoke test that does a simple successful GET, while
you're going about adding your chunked encoding regression test(s)?
Oh, wait: there is a functional test that uses the network, but it's at
the end of httplib.py rather than being part of the test suite, and it
follows the "Guru Watches Output" pattern :-)
I tried to add a test for urllib2 recently that relied on a python.org URL
being set up, but the python.org guys are pretty overworked already and
haven't had time to enable that yet -- so I think that simply from that
point of view your run-your-own-server approach is better. Why does it
need to be unix-y, though? We have SimpleHTTPServer. We're not trying to
test the OS, so I don't see a problem with using loopback and a single
process -- and that would keep test run times down. Um, except that MS
OSes seem a bit odd re the loopback interface -- ISTR that, at least back
with NT4, you just didn't get a loopback i/face unless you had an ethernet
driver installed (back then, I was on dialup).
More unit tests would also be valuable, of course.
More information about the Python-Dev