Utility to screenscrape sites using javascript ?

KB keith at nekotaku.com
Sat Jan 30 20:28:47 CET 2010


> On Sat, 30 Jan 2010 06:21:01 -0800, KB wrote:
> > I have a service I subscribe to that uses javascript to stream news.

> There's a Python interface to SpiderMonkey (Mozilla's JavaScript
> interpreter):
>
> http://pypi.python.org/pypi/python-spidermonkey

Thanks! I don't see a documentation page, but how would one use this?
Would you download the HTML using urllib2/mechanize, then parse for
the .js script and use spider-monkey to "execute" the script and the
output is passed back to python?

TIA.



More information about the Python-list mailing list