read web page that requires javascript on client

[lkcl]

On Mar 18, 8:25 pm, "R. David Murray" <rdmur... at bitdance.com> wrote:
> Greg <gregsaundersem... at gmail.com> wrote:
> > Hello all, I've been trying to find a way to fetch and read a web page
> > that requiresjavascripton the client side and it seems impossible.
> > I've read several threads in this group that say as much but I just
> > can't believe it to be true (I'm subscribing to the "argument of
> > personal incredulity " here).
>
> > Clearly urllib and urllib2 don't seem to support this and I've looked
> > at win32com.client and it's ScriptControl but that doesn't seem to be
> > viable approach for this particular problem.
>
> > Does anyone have any suggestions, hack or ideas or am I missing
> > something really obvious.
>
> Well, this is what is called a Hard Problem :).  It requires not
> only supporting the execution ofjavascript(and therefore an entire
> additional language interpreter!), but also translating that
> execution into something that doesn't have a browser attached to it
> for input or output.
>
> That said, I've heard mention here of something that can apparently be
> used for this.  I think it was some incarnation of Webkit.

 yep.  patch #16401 - don't use the cut-down version that the other
company who are doing "vala" bindings are using - use the version
that  i've worked on, until they support the "full" DOM bindings.
better yet, just grab the code from the git repository i'm maintaining
- http://github.com/lkcl/webkit/tree/16401.master


> I remember
> someone saying

 yep, it was me :)

> you wanted to use the one with, I think it was GTK
> bindings, even though you were dealing with just network IO.  But I don't
> remember clearly and did not record the reference.  Perhaps the person
> who posted that info will answer you,

i do searches for the words "ajax" and "javascript" and "pyjamas"
using groups.google.com occasionally, and pick things up -
eventually :)


>  Unfortunately I'm not 100% sure it was Webkit.

 it was - however i've since found three other projects (including
python-hulahop, the best other alternate candidate).

 l.