[Baypiggies] Adding testing to screen-scraping code
asheesh at asheesh.org
Mon Dec 17 01:30:31 CET 2007
I am a (happy!) customer of a company that provides a service, but I
prefer to automate my interactions with their website, so I wrote a Python
module using mechanize that does that. Now that I have a hunk of code
that works, and it's been working for pretty much two years with minimal
fixes, I decided I'd like to add some tests. I'd like to be able to test
my code without bothering the real web service. First of all, does anyone
I'll say what I'm thinking: It'd be nice to have a mock HTTP server,
maybe, or a mock trivial version of their web app, bundled as part of the
test suite. Then I could easily set a flag in my code to ask it to use
not realwebsite.com but localhost:8118 or some port, and then go through
the usual methods and verify that they work.
It'd be *nicest* if such a thing could be automatically generated from the
calls to urllib2 or mechanize by watching the function calls and noticing
their return values.
Does something like this exist? Is there another angle I should consider?
The reason I want something like this is that the web interface changed in
a tiny way, and then my code broke. But I thought about it, and I wasn't
even sure what used to work before.
FORTUNE PROVIDES QUESTIONS FOR THE GREAT ANSWERS: #15
A: The Royal Canadian Mounted Police.
Q: What was the greatest achievement in taxidermy?
More information about the Baypiggies