[Python-Dev] Googlebot and the mail.python.org python-dev?archive

[A.M. Kuchling]

On Sat, Feb 28, 2009 at 10:37:09AM +0000, Antoine Pitrou wrote:
> have the following HTML tag in them:
>     <META NAME="robots" CONTENT="noindex,follow">
> which explicitly instructs Web spiders *not* to index contents nor follow links.

I believe this makes spiders not index this page, but does follow
links.  Individual messages have "index,nofollow".

--amk