[Python-Dev] Extending xmlrpclib to handle HTTP proxies

[Bill Bumgarner]

On Thursday, December 5, 2002, at 05:31 AM, Fredrik Lundh wrote:
> mal wrote:
>> Doesn't urllib already support this out of the box ? You can use
>> its machinery for finding out the proxy settings and then add them
>> to Transport as needed.
>
> urllib doesn't allow you to override content-types and other things
> that you need to talk proper XML-RPC, and my "request body" [1]
> patch was rejected by GvR.

Yes -- I was surprised to find that urllib doesn't have the ability to 
return the prepared, but no data sent, HTTP instance.   This would seem 
to be very useful.   It would also seem to be easy to do -- split the 
open_http() method into two methods at the 'h = httplib.HTTP(host)' 
line.  The first method would simply return the created HTTP object 
while the second method would send the request, etc...

In any case...

> I'm pretty sure you can use urllib2 for this purpose, though, but I
> haven't looked deeper into that (mostly because people has been
> promising me proxy patches for years...)

... urllib2 seems to be the correct answer for a number of other 
reasons.   In particular, if the implementation is done correctly the 
developer can easily add a Handler to deal with any proxy/communication 
situations that I have not yet imagined (off the top of my head, 
dealing with TCP/IP proxying firewalls...).

Well -- I have pretty much coded myself into a corner out of which the 
only path is to add working proxy support to xmlrpclib.   Our product 
[see www.codefab.com, if curious] includes a first class python based 
XML-RPC client [open source'd soon].  Given that product is aimed at 
the corporate world (among others), the client must be able to deal 
with any existing infrastructure, including HTTP proxies that will so 
often be encountered in such situations.

> (Bill, note that Python CVS isn't the "master repository" for 
> xmlrpclib;
> if you come up with a clean and nice urllib2-based proxy transport,
> you can just mail it to me)

Will do.

> </F>
>
> 1) http://effbot.org/zone/python-urllib-requestbody.htm

thanks,
b.bum