urlretrieve() or urlopen() ??

MCollins at co.seminole.fl.us MCollins at co.seminole.fl.us
Fri Apr 11 14:33:31 CEST 2003


hi,

still working on the utility to download log files via http.

my first solution (which worked) was using urlopen(), reading the logfile
line by line and writing it to a new file line by line.  this felt a little
slow when working with 200mb files across an http connection.

so, i switched to urlretrieve(), thinking that would be faster.  however,
with urlretrieve() i get a "no buffer space available" error.

i can always go back to the urlopen code, because that worked.  however,
any suggestions on ensuring i'm using the fastest process here?

thanks


code:

######
##this is the code that worked
######

def GetUrl(Url,Path):
    http = urllib.urlopen(Url)


    line =  http.readline()
    logfile = open(Path, "w")
    while line != "":
        logfile.writelines(line)
        line = http.readline()
    else:
        logfile.close()


######
###this is the new, broken code
#####

def GetUrl(Url,Path):
    from urllib import urlretrieve, urlcleanup
    urlretrieve(Url,Path)
    urlcleanup()





Matthew Collins
Web Designer/Developer
Seminole County, FL
http://www.co.seminole.fl.us









More information about the Python-list mailing list