internet searching program

Michael Tobis mtobis at
Sat Aug 9 18:07:25 CEST 2008

I think you are talking about "screen scraping".

Your program can get the html for the page, and search for an
appropriate pattern.

Look at the source for a YouTube view page and you will see a string

         var embedUrl = 'http://....

You can write code to search for that in the html text.

But embedUrl is not a standard javascript item; it is part of the
YouTube design. You will be relying on that, and if YouTube changes
how they provide such a string, you will be out of luck; at best you
will have to rewrite part of your code.

In other words, screen scraping relies on the site not doing a major
reworking and on you doing a little reverse engineering of the page

So how to get the html into your program? In python the answer to that
is urllib.


More information about the Python-list mailing list