[Tutor] URLLIB / GLOB

[Kent Johnson]

John wrote:
> Hello,
>  
> I would like to write a program which looks in a web directory for, say 
> *.gif files. Then processes those files in some manner. What I need is 
> something like glob which will return a directory listing of all the 
> files matching the search pattern (or just a simply a certain extension).
>  
> Is there a way to do this with urllib? Any other suggestions?

If the directory is only available as a web page you will have to fetch 
the web directory listing itself with urllib or urllib2 and parse the 
HTML returned to get the list of files. You might want to use 
BeautifulSoup to parse the HTML.
http://www.crummy.com/software/BeautifulSoup/

Kent