[Doc-SIG] docs.python.org, .txt and search engines

[Aahz]

On Fri, Dec 12, 2008, jason kirtland wrote:
>
> The reST sources on http://docs.python.org/ are getting indexed in
> Google and ":mod:`whatnot`" is showing up in search results.  That's
> been more distracting than valuable to me in my searches.  Maybe add a
> robots.txt exclusion for /_sources and the other _sources directories
> on docs.python.org?
> 
>   User-agent: *
>   Disallow: /_sources/
>   Disallow: /3.0/_sources/

This is the first time I've heard this complaint, and nobody else has
spoken up in favor.  Overall, I think I'm -0; somebody may well want to
search the sources.
-- 
Aahz (aahz at pythoncraft.com)           <*>         http://www.pythoncraft.com/

"It is easier to optimize correct code than to correct optimized code."
--Bill Harlan