[Python-Dev] urllib.urlretrieve() and handling 550 errors when reading from FTP

[Brett C.]

Guido van Rossum wrote:
>>Bug #635453 reports that when you use urllib.urlretrieve() to read from 
>>an FTP server and the file you are trying to get does not have the 
>>proper permissions for you, you are given a listing of the directory 
>>instead of an exception (this is all done by Lib/urllib.py:752).
>>
>>Now this doesn't make sense to me and I suspect most people would agree. 
>>  But having that explicit line there on top of it having been in the 
>>file since it was renamed from urlopen way backin 1994 makes me hesitate 
>>before I delete the line.
>>
>>Anyone have issues if I ditch the line so an exception is raised instead 
>>of getting a directory listing instead?
> 
> 
> If you remove this, you won't be able to get a directory listing of
> directories.  I tried this with and without the line; with that line
> in, it gives a directlry listing, without it gives an IOError (Not a
> regular file):
> 
> 
>>>>urllib.urlretrieve("ftp://ftp.python.org/pub", "xyzzy")
> 
> 
> So I'd be against deleting this, unless there's a different way to get
> directory listings.
> 

Try it again but with a trailing slash on the address.  With the line 
removed it will fetch the directory.  Otherwise it errors out.  The code 
apparently tries to get the file and when it fails with a 550 (meaning 
the file cannot be accessed because of permissions or lack of 
existence), the code then send a LIST command which lists whatever the 
argument is, file or directory.

Unfortunately I can't think of a good way to detect the difference 
between a file and a directory that just doesn't have the ending slash 
beyond reading the output and seeing if it is a single listing for a 
file with the same name as the LIST request.

-Brett