Suggestion: "Completing" HTTP support in Python.

[Edward Jason Riedy]

And Oleg Broytmann writes:
 - 
 - GET:  filename, headers = urllib.urlretrieve("http://my.host.name:8080/loginscript?%s" % urllib.urlencode({"param": value}))
 - POST: filename, headers = urllib.urlretrieve("http://my.host.name:8080/loginscript", urllib.urlencode({"param": value}))

You only answered half of his request.  The other half is _not_
supported by urllib: adding HTTP headers.

I believe he's looking for a libwww-perl equivalent in Python.  
I am, too.  LWP is one major reason I still write Perl code on 
a daily basis.  The design is very nice.  You have an LWP::UserAgent 
instance to which you can attach a cookie jar, specialize, etc.  
You form HTTP::Request instances, modify the HTTP parameters, 
and pass them to the UserAgent.  You get back an HTTP::Response, 
which has all sorts of handy methods (is_error, is_success, etc.).  
You can also have the response sent to a callback as it arrives.

It's a very user-friendly design.  The same set of packages 
provides both urllib-like simplicity and advanced features.  
Unfortunately, LWP only supports HTTP/1.0, so it can't harness 
the power of pipelining and chunks.  The LWPng that was supposed 
to replace it seems to be on permanent hold.  It'd be a nice 
thing to see in Python...  See http://www.linpro.no/lwp/ for more.

Oh, and LWP code is somewhat more legible than the examples you've
provided, imho, but also can be more verbose.  The easy version:

  use LWP::Simple; use URI::Escape;
  $doc = get "http://my.host.name:8080/loginscript?param=" .
             uri_escape($value);

More complicated:

  use LWP::UserAgent; use URI;

  $ua = new LWP::UserAgent;
  $url = URI->new("http:://my.host.name:8080/loginscript");
  $url->query_form("param" => $value);

  $req = new HTTP::Request 'GET' => $url;
  $req->header('Accept' => 'text/html');
  $res = $ua->request($req);

  if ($res->is_success) {
  # ...

Perl borrowed from Python's OO model, so borrowing from a few 
Perl modules is fair...  ;)

Jason