Python Screen Scraper

James Stroud jstroud at
Tue Apr 24 13:50:07 CEST 2007


Does anyone know of an example, however modest, of a screenscraper 
authored in python? I am using Firefox.

Basically, I am answering problems via my browser and being scored for 
each problem. I have a tendency to go past my peak for training 
efficiency, so I would like to scrape the result page for each problem I 
answer, compile statistics, and have a program alert me when I should 
stop (based on score and accuracy--assuming training value is related to 
changes in these metrics).

I have no idea how to go about writing such a beast and I am hoping that 
I could get some pointers or an example that could get me going in the 
right direction.

Parsing, etc, is not a problem, but I'm not exactly sure how I might 
interface python with Firefox, forwarding scraped pages to my browser 
(or forwarding from the browser to the scraper).

Thanks in advance for any help or advice.


More information about the Python-list mailing list