[Tutor] python module to search a website

[Corey Richardson]

On 02/26/2011 10:11 PM, vineeth wrote:
> Hello all,
> 
> I am looking forward for a python module to search a website and extract 
> the url.

What website, what is it searching for, and what URL is it looking for?

> 
> For example I found a module for Amazon with the name "amazonproduct", 
> the api does the job of extracting the data based on the query it even 
> parses the url data. I am looking some more similar query search python 
> module for other websites like Amazon.

The only module I found for amazon-product was a python interface to
Amazon's advertising API. What data does it extract, what query, and
which URL does it parse? From what I found that module uses the API to
search the website, a service provided by Amazon and not something
Python is doing itself.

You may want to look into urlparse and urllib2, for parsing URLs and
opening websites respectively.

http://docs.python.org/library/urlparse.html
http://docs.python.org/library/urllib2.html

If that isn't what you're looking for, you'll need to be a bit more
descriptive.

If you are going to be parsing the HTML and then searching for specific
elements you might look into BeautifulSoup.

-- 
Corey Richardson