Merlin, a fun little program

Ron Stephens rstephens at
Mon Jul 7 23:05:28 CEST 2003

Oren wrote """You can still use Google for this """

Yes indeed. I have played with the Google API's, registered and also
use pygoogle. They make this kind of thing easier, no doubt about it.
The reason I used my hand-rolled web scraper on Yahoo is that using
the Google API's means that other potential users, like those who
download form my web site, can't run my code it uses Google api's
unless they download and register also; which might be pain for them.

At any rate, doing my own was fun and informative for me. A big
disadvantage to web scraping is that they code tends to break over
time, though. This happened to me with Max M's original; two year old
algorithm. Google broke it, and I didnt realize it unit I can back and
retried the code.

The two links you gave are good ones and I studied both in my efforts;
I recommend them. Thanks for the inputs.

I guess the bigger question is; is there anything wrong with web
scraping? I surely never meant any harm in it, and certainly no money
is involved. But maybe I should give it up and do other things?

Ron Stephens

More information about the Python-list mailing list