[Tutor] Data Gathering

Kent Johnson kent37 at tds.net
Mon Aug 20 02:20:49 CEST 2007

Johnny Jelinek IV wrote:
> Hi,
> I was wondering if something like this is possible; Can I create a 
> python script that will connect to a website to use it's search features 
> to gather information for me?  For example, if I wanted information 
> about a movie from imdb, or something from wikipedia, but didn't want to 
> go to the website, could I create a program to let me search and it 
> would output data from those sites to me in my program?

For IMDB specifically there are a couple of Python packages that will 
fetch the data for you.

In general, you will have to learn about form submission using urllib or 
urllib2 and HTML parsing with perhaps BeautifulSoup. A few links to help:



More information about the Tutor mailing list