[ python-Bugs-1016880 ] urllib.urlretrieve silently truncates downloads

[SourceForge.net]

Bugs item #1016880, was opened at 2004-08-26 08:58
Message generated for change (Comment added) made by rhettinger
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1016880&group_id=5470

Category: Python Library
Group: None
Status: Open
Resolution: None
Priority: 6
Submitted By: David Abrahams (david_abrahams)
Assigned to: Nobody/Anonymous (nobody)
Summary: urllib.urlretrieve silently truncates downloads

Initial Comment:
The following script appears to be unreliable on all 
versions of Python we can find.  The file being 
downloaded is approximately 34 MB.  Browsers such as 
IE and Mozilla have no problem downloading the whole 
thing.

----

import urllib
import os

os.chdir('/tmp')
urllib.urlretrieve
('http://cvs.sourceforge.net/cvstarballs/boost-
cvsroot.tar.bz2',
                  'boost-cvsroot.tar.bz2')


----------------------------------------------------------------------

>Comment By: Raymond Hettinger (rhettinger)
Date: 2004-08-26 15:04

Message:
Logged In: YES 
user_id=80475

Followed the same procedure (no chdir, add a hook) but
bombed out at 9.1Mb:

 . . .
(1117, 8192, 34520156)
('boost-cvsroot.tar.bz2', <httplib.HTTPMessage instance at
0x00B1E4B8>)

----------------------------------------------------------------------

Comment By: Tim Peters (tim_one)
Date: 2004-08-26 13:52

Message:
Logged In: YES 
user_id=31435

Hmm.  I don't know anything about this, but thought I'd just 
try it.  Didn't chdir(), did add a reporthook:

def hook(*args):
    print args

WinXP Pro SP1, current CVS Python, cable modem over a 
wireless router.  Output looked like this:

(0, 8192, 34520156)
(1, 8192, 34520156)
(2, 8192, 34520156)
...
(4213, 8192, 34520156)
(4214, 8192, 34520156)
(4215, 8192, 34520156)

Had the whole file when it ended:

> wc boost-cvsroot.tar.bz2
 125368  765656 34520156 boost-cvsroot.tar.bz2

*Maybe* adding the reporthook changed timing in some 
crucial way.  Don't know.

----------------------------------------------------------------------

Comment By: Raymond Hettinger (rhettinger)
Date: 2004-08-26 12:09

Message:
Logged In: YES 
user_id=80475

Confirmed.  On Py2.4 (current CVS), I got 12.7 Mb before the
connection closed.

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1016880&group_id=5470