web scrapping - POST and auto-login

james <at> hal-pc.org james <at> hal-pc.org
Tue Sep 20 06:25:44 CEST 2005

What i have so far:

from urllib import urlopen
from ClientForm import ParseResponse
from re import search

URL = "http://www.hal-pc.org/networking/cgi-bin/wepgen.cgi"
pattern = "<!-- marker -->*([0-9,a-f,]*)"
doc = urlopen(URL).read()
result = search(pattern, doc)
wep64 = result.group(1)[:10]
#print wep64

forms = ParseResponse(urlopen("http://un:pw@wifirouter/WL_WPATable.asp"))
form = forms[0]
form["wl_key1"] = wep64
#print form
response = urlopen(form.click("save_button"))

URL = "http://un:pw@wifirouter/WL_WPATable.asp"
pattern2 = "wl_key1 value='*([0-9,a-f,]*)"
doc2 = urlopen(URL).read()
result2 = search(pattern2, doc2)
#print result2.group(1)

the only problems now are getting it to see the "save_button" correctly or 
generate the POST from scratch as Max suggested..  that will take some more 
hunting to figure that out...  any better phrases to search for maybe?
and email it out.  can probably "borrow" some things from Freevo on calling 
another program, just passing the variable to it as a PIPE is going to be the 
hard part.  thinking of using sendmail (not a problem), but how to get the data? 
  i'll keep looking.  thanks again both of you.

More information about the Python-list mailing list