[Tutor] Web Harvesting & AJAX/Javascript

[Walter Prins]

Hello Roy

On 29 November 2010 19:42, Roy Hinkelman <royhink at gmail.com> wrote:

> Researching this has led me to PAMIE and Selenium. PAMIE is giving me
> problems with permissions, and the discussion group appears to be fading
> away. I have not tried Selenium yet. Both open a browser instance, and PAMIE
> is quite slow, and I expect Selenium to be quite slow as well.
>
> How are you navigating around these Javascript based pages? Is there a
> library or plugin that I am missing?
>

I'm no expert in this area, however I have done some research in this
direction in the past, also to deal with sites that use Javascript
(obviously Python is very adept at interacting with sites that are plain
HTML by itself, e.g. there's lxml, BeautifulSoup & friends etc) and I remain
interested in this.

Anyway so then, here's my $0.02 for what it's worth.  Firstly, Selenium may
be an option, I briefly played with it quite some time ago, I don't know
PAMIE.

However, I suspect what you might rather want to look at is perhaps
something like the PyWebkitGTK (or PyWebkitQT) which leverages the webkit
browser engine.  (This kind of assumes you're on Linux... are you?)

Some hopefully relevant links:
http://webkit.org/
http://code.google.com/p/pywebkitgtk/
http://www.gnu.org/software/pythonwebkit/
http://www.pygtk.org/pygtkmozembed/
http://directfb.org/

Best wishes,

Walter
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/tutor/attachments/20101201/d8679f04/attachment.html>