Download file from web site

[Gerhard Häring]

On Mon, Dec 03, 2001 at 10:03:13PM -0700, Thomas Pham wrote:
> Hi group,
> 
> I am trying to download a file from a website.   My source code is
> listed below.    But I can only get to its first page.   I could not
> go further to its second page.    The second page "idm_download"
> however, does require an user ID & password to open it.

I'd try my luck with urllib2. The module documentation contains an
example for using HTTP digest authentication. Oups. That's obviously one
of those horrible underdocumented modules in the Python standard
library. And looks a little bit overdesigned, too. Argh!

Let's try good 'ole urllib instead. Here's an example that works for a
page needing authentication:

import urllib
    
class MyOpener(urllib.FancyURLopener):
    def prompt_user_passwd(self, host, realm):
        # unconditionally return the correct username, password tuple
        return ("ghaering", "mysuperduperpassword")


opener = MyOpener()
print opener.open("http://www.dmoz.org/editors").read()

Gerhard
-- 
mail:   gerhard <at> bigfoot <dot> de       registered Linux user #64239
web:    http://www.cs.fhm.edu/~ifw00065/    OpenPGP public key id 86AB43C0
public key fingerprint: DEC1 1D02 5743 1159 CD20  A4B6 7B22 6575 86AB 43C0
reduce(lambda x,y:x+y,map(lambda x:chr(ord(x)^42),tuple('zS^BED\nX_FOY\x0b')))