JavaScript web scraping test cases?

John J Lee jjl at pobox.com
Sat Aug 23 10:24:15 EDT 2003


On Fri, 22 Aug 2003, Skip Montanaro wrote:

>
>     >> Anyway, the point of this post is that I'm looking for pages to test
>     >> it on, so if you have a page that you'd like scraped (one that uses
>     >> JavaScript in some non-trivial way, of course! ...
>
>     John> Nobody?
>
> Sorry, I couldn't think of anything off the top of my head.  In my own pages
[...]

Oh, I'm sure I'll have no trouble finding test cases -- I just thought
that, rather than some random sites that are of no use to anyone, there is
bound to be somebody out there who actually wanted to scrape a particular
page in the past, and had not bothered previously thanks to the
inconvenience of having to read & reproduce the effect of the JS code
(particularly code that messes about with forms).  It would be nice to be
doing something useful at the same time as writing tests!

Of course, I already have those sites that gave rise to the 'itch' to do
this in the first place, but I'm sure there's lots of the browser object
model that they don't exercise...


John





More information about the Python-list mailing list