Javascript website scraping using WebKit and Selenium tools

Veek M vek.m1234 at gmail.com
Thu Jul 2 08:31:46 CEST 2015


dieter wrote:

> Once the problems to get the "final" HTML code solved,
> I would use "lxml" and its "xpath" support to locate any
> relevant HTML information.

Hello Dieter, yes - you are correct. (though I don't think there's any auth 
to browse - nice that you actually tried) He's using jsonP and updating his 
html. I decided to manually mangle it.

urllib to download, re to nuke the jsonp(".........stuff i want......") and 
then lxml. It works and I got the text. Now i need to translate - many 
thanks.

I should have checked first using HTTP Headers to see what he was 
downloading - i'm an ass. Oh well solved :)


More information about the Python-list mailing list