[Tutor] URLLIB / GLOB

Kent Johnson kent37 at tds.net
Tue Oct 23 03:09:26 CEST 2007


John wrote:
> Hello,
>  
> I would like to write a program which looks in a web directory for, say 
> *.gif files. Then processes those files in some manner. What I need is 
> something like glob which will return a directory listing of all the 
> files matching the search pattern (or just a simply a certain extension).
>  
> Is there a way to do this with urllib? Any other suggestions?

If the directory is only available as a web page you will have to fetch 
the web directory listing itself with urllib or urllib2 and parse the 
HTML returned to get the list of files. You might want to use 
BeautifulSoup to parse the HTML.
http://www.crummy.com/software/BeautifulSoup/

Kent


More information about the Tutor mailing list