The webdownload.py script in Abe Fettig's book works fine for directly connected machines. I would like to modify it to use a proxy. Most of the examples in the twisted docs seem to show how to write a proxy, not how to use one. To modify that example to use a proxy, do I need to use a ProxyClientFactory instead of the HTTPDownloader? I tried that, but it needs 7 arguments, and I could not tell what they should be. Or do I just set the factory.protocol to proxy.Proxy? If so, where do I tell it the proxy's address & port? Is there a (simple(!)) twisted web page retrieval script that uses (i.e. runs behind) a proxy server that I could use to start with? Thanks, Wendell
For web2, I have written an implementation of iweb.IResource that generates the result of its renderHTTP method via a proxying client access to another HTTP server. See ProxyResource at http://foss.eepatents.com/DynamicSite/browser/trunk/dynamicsite/external.py Best regards, Ed Wendell Turner wrote:
The webdownload.py script in Abe Fettig's book works fine for directly connected machines. I would like to modify it to use a proxy. Most of the examples in the twisted docs seem to show how to write a proxy, not how to use one.
To modify that example to use a proxy, do I need to use a ProxyClientFactory instead of the HTTPDownloader? I tried that, but it needs 7 arguments, and I could not tell what they should be.
Or do I just set the factory.protocol to proxy.Proxy? If so, where do I tell it the proxy's address & port?
Is there a (simple(!)) twisted web page retrieval script that uses (i.e. runs behind) a proxy server that I could use to start with?
Thanks,
Wendell
_______________________________________________ Twisted-web mailing list Twisted-web@twistedmatrix.com http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-web
On Mon, Aug 28, 2006 at 04:29:31PM -0400, Wendell Turner wrote:
The webdownload.py script in Abe Fettig's book works fine for directly connected machines. I would like to modify it to use a proxy. Most of the examples in the twisted docs seem to show how to write a proxy, not how to use one.
So that everyone is on the same page, are you attempting to download pages via an http proxy? If yes: There is currently a ticket in the tracker about client support for proxies http://twistedmatrix.com/trac/ticket/1774 see his patch and the notes about his patch for help. It would not be difficult to clean up his patch and make it ready for inclusion but I have no time to do this and I don't suspect he has time either. If no: Do you want to set up a reverse proxy or a full http proxy? If reverse: See twisted.web.proxy.ReverseProxyResource and associated documentation If Full HTTP proxy: See the rest of twisted.web.proxy -- "Usually the protocol is this: I appoint someone for a task, which they are not qualified to do. Then, they have to fight a bear if they don't want to do it." -- Glyph Lefkowitz
On Mon, Aug 28, 2006 at 02:33:02PM -0700, David Reid wrote:
On Mon, Aug 28, 2006 at 04:29:31PM -0400, Wendell Turner wrote:
The webdownload.py script in Abe Fettig's book works fine for directly connected machines. I would like to modify it to use a proxy. Most of the examples in the twisted docs seem to show how to write a proxy, not how to use one.
So that everyone is on the same page, are you attempting to download pages via an http proxy?
Yes, just using an http proxy. Current solution is: def gotProtocol(p): p.sendMessage("GET " + webpageurl + "\n\r") c.connectTCP(proxyhost, proxyport).addCallback(gotProtocol) But there should be a better way.
If yes: There is currently a ticket in the tracker about client support for proxies http://twistedmatrix.com/trac/ticket/1774
Thanks, I'll take a look at the patch. Although, according to the notes, it may take some effort to do it 'properly'. Wendell
participants (3)
-
David Reid -
Ed Suominen -
Wendell Turner