http client with proxy example ?

[Jaroslav Gergic]

Kevin Bailey wrote:
> 
> Hi,
> 
> Could someone point me to or provide a simple example of
> pulling down a webpage using a proxy webserver in Python ?
> 
I can not help "in Python" but basically, there is simple difference
between HTTP request and HTTP-proxy request:

HTTP: you want to get http://some-host/some-file.html

open socket to "some-host"  port 80 (default) and issue:

>>>BEGIN_SAMPLE:<<<
GET /some-file.html HTTP/1.0
Host: some-host

>>>END SAMPLE<<<

(do not forget an extra cr+lf after Host header)

HTTP-proxy: you want to get http://some-host/some-file.html
using proxy: some-proxy:8080 (8080 is typical but can differ)

open socket to "some-proxy" port 8080 and issue:
(connect to PROXY instead of target server!)

>>>BEGIN_SAMPLE:<<<
GET http://some-host/some-file.html HTTP/1.0
Host: some-host

>>>END SAMPLE<<<

Summary: the only difference is in GET header line,
in case of HTTP-proxy request you MUST issue fully qualified URL.

Regards
Jaroslav Gergic

P.S.: You can try the examples above using Telnet ;)