python and web pages

Diez B. Roggisch deets at nospam.web.de
Thu Nov 19 09:43:50 CET 2009


Daniel Dalton schrieb:
> Hi,
> 
> Here is my situation:
> I'm using the command line, as in, I'm not starting gnome or kde (I'm on
> linux.)
> I have a string of text attached to a variable,. So I need to use one of
> the browsers on linux, that run under the command line, eg. lynx,
> elinks, links, links2 and do the following.
> 1. Open a certain web page and find the first text box on the page, and
> put this text into the form.
> 2. Press the submit button, and wait for the result page to load.
> 3. Click on the 15th link down the page.
> 
> So, how do I do this, can anyone point me to some docs or modules that
> may help out here?

Use the module "mechanize". It will mimic a browser, and allows you to 
navigate to pages, fill in forms & submit them.

And with lxml or BeatifulSoup, you can extract the link you need to 
press, and then make mechanize visit it.

Diez



More information about the Python-list mailing list