[Tutor] Web browser

Kent Johnson kent37 at tds.net
Mon Jun 13 12:05:57 CEST 2005

Liam Clarke wrote:
> Can Beautiful Soup execute scripts? I was trying to scrape using 
> urllib2, and the Javascripts messed my navigation up something chronic.

No, Beautiful Soup won't run scripts. AFAIK the only way to access the web from Python and execute JavaScripts is to automate a browser and use the browser's JavaScript engine. (e.g. Pamie, COM, Selenium http://confluence.public.thoughtworks.org/display/SEL/Home)

If Java/Jython is an option you could try HttpUnit which includes a JS interpreter.


More information about the Tutor mailing list