[Tutor] how to set time limit before downloading file

[wesley chun]

On 8/24/06, kulibin da-da! <kylibine at gmail.com> wrote:
> I had to download some files.
>
>  Using the code below I could download almost all I had to download, but I
>  have a problem
>    :
> r = urllib.urlopen(url)
>    :
> And the problem is that the script sleeps for some hours. And thats
> all - no data is read.


is there a reason why you are using urlopen() instead of
urlretrieve(url, localFile)?  it will download the file for you so you
don't have to do it chunks at a time.  is it because you're working
with an unreliable server where a timeout is required? otherwise yes,
you will have to set a socket timeout to give up on a partial download
if the other side is hanging.

you may also want to consider multithreading this application (using
the threading module). if the individual downloads are not dependent
on each other.  I/O-bound applications are a great use case for Python
MT and will help improve performance. it will allow for multiple
"simultaneous" downloads in parallel and allow it to complete sooner.

cheers,
-- wesley
- - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
"Core Python Programming", Prentice Hall, (c)2007,2001
    http://corepython.com

wesley.j.chun :: wescpy-at-gmail.com
python training and technical consulting
cyberweb.consulting : silicon valley, ca
http://cyberwebconsulting.com