*** Probably not a problem with Twisted... see my last comments ***
Well, any open source tester, even a simple dodgy one you write yourself -- I don't have any strong preference, but it would be nice to be able to reproduce your results :)
Well, I've attempted to write a script in python to do this, but, I'm not seeing the performance difference when I run it (It performs well, whether it's 1.0 or 1.1. Actually, it performs slightly BETTER if it's 1.1, but only about 10%)! After rubbing my eyes, and trying again (and again, and again....) I've concluded that I'm NOT losing my mind (the perf degradation is very real and noticable when loading/refreshing a page with lot's of small images), but for some reason, the way httplib constructs an HTTP/1.1 request doesn't trigger it (while ACT, Internet Explorer, and Mozilla 1.7b do trigger it)
I can't think of what else to suggest, except for running the server in the Python profiler (the -p=profile.log switch to twistd, if you're using it), and seeing if that reveals where the extra time is being spent.
Hmm... I'm not running twistd, but maybe I can do something along those lines in my script that kicks Twisted off... It already has some profiling bootstrap code in it. I hate wading through all the profiler output, but at least this may be easy to spot, since it's so dramatic.
Maybe you could insert a "print repr(line)" into HTTPChannel's lineReceiver handler (see twisted/protocols/http.py) to get a more raw dump of the request, and double-check that there's no significant differences in the HTTP/1.0 vs. HTTP/1.1 requests, but I'm doubtful...
Another good suggestion. I don't know where that is at the moment, but I'm sure it'll be easy to find once I dig/grep for it. I assume that's a pretty low level spot in twisted's comm stack?
Also, to eliminate another difference between your environment and mine, is it possible to run your tester against a Twisted server on a linux box, and verify that it still gives the same behaviour? If for some bizarre reason it turns out to be a windows-only issue, it'd be good to know before I waste too much time trying to reproduce it on Linux :)
Actually, I am running it on a Linux box. Of course I ran ACT from a windows box, but they're sitting next to each other on a 100mbps lan. The data transfer rate was not a factor (Apache on Linux can hand about 500 images / second to the windows box). I'm using Python 2.3 on SuSE 8.1 with a 2.4.24 kernel (I compiled it myself, but I haven't had any troubles with it -- that I know of ;) )
Also, I'm losing track of what you have and haven't tried... it might be time we opened a bug report for this, so we can track this properly.
I'd be up for that, but I'd really like to see at least one other person duplicate it before we went that far. I may just have something stupid going on over here. ---------------------------------------------------------------- Ok, you know what? I'm starting to think it's a not a problem intrinsic with twisted. I just put together a html page with 100 small images, and served it through the Qx / Twisted combo and it was slow. Through Qx / Medusa, it was fast. Through Twisted by itself, it was fast (using Moz 1.7b). Here's the script I used:
#!/usr/bin/python from twisted.internet import reactor from twisted.web import static, server
root = static.File( "/home/jsibre/programs/python/sitehostApp/sitehost/web/static/test") reactor.listenTCP(8082, server.Site(root)) reactor.run()
It ran really fast, all images loaded in about a second (through the Qx / Twisted combo, you can watch them load one after the other... Takes about 8 or 9 seconds).
I was afraid of this... It's probably something in the glue that ties the Qx to Twisted. I'll work it from that angle for a while and see if I find a problem that is fixable, and provide the fix to the Qx folks when I figure it out.
Thanks for your help, Andrew.
Jason