read web page that requires javascript on client

Greg gregsaundersemail at
Fri Mar 20 02:09:28 CET 2009

On Mar 18, 7:25 pm, Carl <tg2.u... at> wrote:
> On Mar 18, 1:56 pm, a... at (Aahz) wrote:
> > In article <mailman.2143.1237407931.11746.python-l... at>,
> > R. David Murray <rdmur... at> wrote:
> > >That said, I've heard mention here of something that can apparently be
> > >used for this.  I think it was some incarnation of Webkit.  I remember
> > >someone saying you wanted to use the one with, I think it was GTK
> > >bindings, even though you were dealing with just network IO.  But I don't
> > >remember clearly and did not record the reference.  Perhaps the person
> > >who posted that info will answer you, or you will be able to figure out
> > >from these clues.  Unfortunately I'm not 100% sure it was Webkit.
> > By the power of Gooja!
> >
> > --
> > Aahz (a... at           <*>
> > "Programming language design is not a rational science. Most reasoning
> > about it is at best rationalization of gut feelings, and at worst plain
> > wrong."  --GvR, python-ideas, 2009-3-1
> Probably the easiest thing is to actually use a browser. There are
> many examples of automating a browser via Python. So, you can
> programmatically launch the browser, point it to the JavaScript
> afflicted page, let the JS run and grab the page source. As an added
> bonus you can later interact with the page by programatically, filling
> form fields, selecting options from lists and clicking buttons.
> HTH, Carl

Selenium. It's not pretty for what I want to do but it works ... then
again, what I need to do is not pretty either.

More information about the Python-list mailing list