23 Jun
2000
23 Jun
'00
2:01 p.m.
On Fri, 23 Jun 2000 12:43:20 -0400 (EDT) Jeremy Hylton <jeremy@beopen.com> wrote:
Speaking as a non-expert, these changes sound good reasonable. Do robots other than htdig understand these meta tags?
My experience, from watching my Apache logs (I'm very well indexed), is that ALL of the commercial search engine spiders and robots honour both robots.txt and the robots meta tag.
-- J C Lawrence Home: claw@kanga.nu ----------(*) Other: coder@kanga.nu --=| A man is as sane as he is dangerous to his environment |=--