[pydotorg-www] robots.txt make web archive inaccessible

[anatoly techtonik]

On Wed, Jun 25, 2014 at 11:13 AM, M.-A. Lemburg <mal at egenix.com> wrote:
> On 25.06.2014 06:16, anatoly techtonik wrote:
>> http://web.archive.org/web/*/http://docs.python.org/release/2.2.3/whatsnew/node5.html
>
> Can you propose an alternative robots.txt file ?
>
> """
> # Prevent development and old documentation from showing up in search results.
> User-agent: *
> # Disallow: /dev
> Disallow: /release
> """

Try this:
"""
User-agent: ia_archiver
Disallow:

User-agent: *
Disallow: /release
"""