FTP with urllib2 behind a proxy

[John J. Lee]

pythonguy at Hotpop.com (Anand Pillai) writes:

> You need to install a proxyhandler, authhandler,
> ftphandler and httphandler. Then build yourself
> an opener, that opens the doors for you ... :-)
> 
> The following does the trick.
> 
> proxy_handler = urllib2.ProxyHandler( {'http': 'myhttpproxy:80',
>                                        'https' : 'myhttpsproxy:443',
>                                        'ftp'    : 'myftpproxy:21' } )
> 
> opener= urllib2.build_opener(proxy_handler, urllib2.HTTPBasicAuthHandler(),
>                              urllib2.HTTPHandler, urllib2.HTTPSHandler,
>                              urllib2.FTPHandler)
> 
> # install this opener
> urllib2.install_opener(opener)
> 
> # Go ahead, knock knock!
> 
> req=urlli2.Request('ftp://ftp.gnu.org')
> data=urllib2.urlopen(req).read()

A couple of things to add: you don't need to add handlers that already
get added by default by build_opener (FTPHandler and HTTPHandler, for
example).  ProxyHandler is one of these default handlers, so if your
environment is set up for it (http_proxy, etc. environment variables),
you don't need to supply a ProxyHandler (of course, if your
environ. *isn't*, then you do need to supply one, to give it the proxy
details).  You don't need Request objects (unless you want to add
headers to a Request, or pass Requests around).  You don't need to
install a global opener, unless your code expects it -- it's just a
convenience (or an inconvenience, sometimes).

Actually, did the OP say proxy basic auth. was involved?  Don't
recall.  I've never needed it for proxies, but there seems to be a
ProxyBasicAuthHandler in urllib2, so I guess that's what you meant to
use, rather than HTTPBasicAuthHandler (which is for website auth., not
proxy auth).

So, after all that, you end up with:

opener = urllib2.build_opener(urllib2.ProxyBasicAuthHandler)
data = opener.open('ftp://ftp.gnu.org').read()

(I like to close the response explicitly, though)


John