read web page that requires javascript on client

[R. David Murray]

Greg <gregsaundersemail at gmail.com> wrote:
> Hello all, I've been trying to find a way to fetch and read a web page
> that requires javascript on the client side and it seems impossible.
> I've read several threads in this group that say as much but I just
> can't believe it to be true (I'm subscribing to the "argument of
> personal incredulity " here).
> 
> Clearly urllib and urllib2 don't seem to support this and I've looked
> at win32com.client and it's ScriptControl but that doesn't seem to be
> viable approach for this particular problem.
> 
> Does anyone have any suggestions, hack or ideas or am I missing
> something really obvious.

Well, this is what is called a Hard Problem :).  It requires not
only supporting the execution of javascript (and therefore an entire
additional language interpreter!), but also translating that
execution into something that doesn't have a browser attached to it
for input or output.

That said, I've heard mention here of something that can apparently be
used for this.  I think it was some incarnation of Webkit.  I remember
someone saying you wanted to use the one with, I think it was GTK
bindings, even though you were dealing with just network IO.  But I don't
remember clearly and did not record the reference.  Perhaps the person
who posted that info will answer you, or you will be able to figure out
from these clues.  Unfortunately I'm not 100% sure it was Webkit.

--
R. David Murray           http://www.bitdance.com