[Baypiggies] Adding testing to screen-scraping code

[Asheesh Laroia]

I am a (happy!) customer of a company that provides a service, but I 
prefer to automate my interactions with their website, so I wrote a Python 
module using mechanize that does that.  Now that I have a hunk of code 
that works, and it's been working for pretty much two years with minimal 
fixes, I decided I'd like to add some tests.  I'd like to be able to test 
my code without bothering the real web service.  First of all, does anyone 
have advice?

I'll say what I'm thinking: It'd be nice to have a mock HTTP server, 
maybe, or a mock trivial version of their web app, bundled as part of the 
test suite.  Then I could easily set a flag in my code to ask it to use 
not realwebsite.com but localhost:8118 or some port, and then go through 
the usual methods and verify that they work.

It'd be *nicest* if such a thing could be automatically generated from the 
calls to urllib2 or mechanize by watching the function calls and noticing 
their return values.

Does something like this exist?  Is there another angle I should consider?

The reason I want something like this is that the web interface changed in 
a tiny way, and then my code broke.  But I thought about it, and I wasn't 
even sure what used to work before.

Thanks, all!

-- Asheesh.

--
FORTUNE PROVIDES QUESTIONS FOR THE GREAT ANSWERS: #15
A:	The Royal Canadian Mounted Police.
Q:	What was the greatest achievement in taxidermy?