Robots

[Ben Ocean]

At 05:49 PM 2/15/2001 +0000, you wrote:
>So, getting the emails automatically won't be much help. After all, you'll
>collect many adresses from people other than the relevant webmaster, and
>you'll have to look at the pages manually anyway, to decide who you should
>send your request to.
>
>Although you might make a program to find out which pages have email
>adresses. If you're writing some sort of spider, check out
>Tools/webchecker/webchecker.py in the source distribution, a program that
>checks links on a website; you might be able to adapt it.
>
>So what's left is finding the interesting pages. That's a very hard problem,
>do you have any idea how you want to decide what constitutes a web page
>similar to yours? If you're looking for certain keywords, it may be easiest
>to just enter them into Google...

Yeah. The way Zeus operates, you enter 100 keyword phrases and it looks for 
relevant matches. You could well be right that just hitting Google, or what 
have you, would be just as effective: I've thought the same. Maybe I'll do 
just that. If not, I should at least salt my robot-generated results with 
that. Or better yet, feed that info into my robot. Thanks!
BenO